02:47:43,523 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:47:44,57 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3684828346331961964] 02:47:44,80 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:47:44,273 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:47:44,273 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:47:44,342 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:47:44,345 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:47:44,684 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:47:44,686 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:47:44,708 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:44,712 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:44,714 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:47:45,662 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testBackupDatastore 02:47:45,801 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#-800981009] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:45,812 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:47:45,839 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:47:45,843 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testBackupDatastore 02:47:45,852 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 02:47:45,893 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testBackupDatastore 02:47:45,896 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:47:45,896 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 02:47:45,897 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 02:47:45,926 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@76b96935 02:47:45,938 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} started 02:47:45,946 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:47:45,947 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testBackupDatastore 02:47:45,948 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 02:47:45,948 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore to akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 02:47:45,949 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testBackupDatastore on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore on shard member-1-shard-cars-config_testBackupDatastore whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-24374518] 02:47:45,949 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testBackupDatastore 02:47:45,950 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-24374518] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:45,951 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 02:47:45,951 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore to akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 02:47:45,951 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testBackupDatastore on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore on shard member-1-shard-people-config_testBackupDatastore whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-2023403820] 02:47:45,952 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-2023403820] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:45,972 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testBackupDatastore, persistent : true 02:47:45,972 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-people-config_testBackupDatastore, persistent : true 02:47:46,0 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#-800981009] 02:47:46,2 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:47:46,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 02:47:46,54 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#-800981009] 02:47:46,55 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:47:46,55 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 02:47:46,106 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#-800981009] 02:47:46,106 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:47:46,106 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 02:47:46,110 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#-1133942806] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:46,110 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#-991573132] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:46,111 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#-991573132 created and ready for shard:member-1-shard-cars-config_testBackupDatastore 02:47:46,111 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#-1133942806 created and ready for shard:member-1-shard-people-config_testBackupDatastore 02:47:46,136 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/shard-member-1:snapshot-read#2050081077] 02:47:46,137 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/shard-member-1:snapshot-read#2050081077] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:46,142 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-people-config_testBackupDatastore with journal batch size 1 02:47:46,145 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/shard-member-1:snapshot-read#-659462808] 02:47:46,146 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testBackupDatastore with journal batch size 1 02:47:46,146 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/shard-member-1:snapshot-read#-659462808] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:46,158 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#-800981009] 02:47:46,158 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:47:46,158 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 02:47:46,160 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:46,161 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:46,171 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:46,171 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:46,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 02:47:46,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testBackupDatastore is initialized 02:47:46,173 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 02:47:46,173 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testBackupDatastore is initialized 02:47:46,187 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 02:47:46,188 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 02:47:46,189 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testBackupDatastore is no longer the leader 02:47:46,190 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from null to Follower 02:47:46,190 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testBackupDatastore is no longer the leader 02:47:46,190 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from null to Follower 02:47:46,201 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 02:47:46,201 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 02:47:46,205 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 02:47:46,205 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 02:47:46,206 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-cars-config_testBackupDatastore 02:47:46,206 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-people-config_testBackupDatastore 02:47:46,208 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#-800981009] 02:47:46,209 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:47:46,209 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-24374518] 02:47:46,210 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Starting new election term 1 02:47:46,210 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Starting new election term 1 02:47:46,210 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:47:46,210 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:47:46,211 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 02:47:46,211 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 02:47:46,211 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testBackupDatastore is no longer the leader 02:47:46,211 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testBackupDatastore is no longer the leader 02:47:46,211 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Follower to Candidate 02:47:46,211 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Follower to Candidate 02:47:46,235 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Received ElectionTimeout 02:47:46,236 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Received ElectionTimeout 02:47:46,245 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Leader): Election: Leader has following peers: [] 02:47:46,245 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Leader): Election: Leader has following peers: [] 02:47:46,245 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:47:46,246 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:47:46,246 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 02:47:46,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 02:47:46,249 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: became leader with frontend state for [] 02:47:46,249 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: became leader with frontend state for [] 02:47:46,250 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Candidate to Leader 02:47:46,250 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Candidate to Leader 02:47:46,257 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , registered listener akka://cluster-test/user/shardmanager-config_testBackupDatastore 02:47:46,257 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , registered listener akka://cluster-test/user/shardmanager-config_testBackupDatastore 02:47:46,258 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testBackupDatastore, leaderId=member-1-shard-cars-config_testBackupDatastore, leaderPayloadVersion=13] 02:47:46,262 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testBackupDatastore from Candidate to Leader 02:47:46,262 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testBackupDatastore, leaderId=member-1-shard-people-config_testBackupDatastore, leaderPayloadVersion=13] 02:47:46,262 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testBackupDatastore from Candidate to Leader 02:47:46,262 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testBackupDatastore is ready 02:47:46,308 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#-800981009] 02:47:46,309 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 02:47:46,309 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-2023403820] 02:47:46,310 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:46,321 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:46,332 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testBackupDatastore 02:47:46,342 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore#-70886442] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:46,346 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:47:46,348 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 02:47:46,349 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:47:46,350 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testBackupDatastore 02:47:46,350 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} started 02:47:46,351 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testBackupDatastore 02:47:46,351 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:47:46,352 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 02:47:46,352 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 02:47:46,352 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@190e73b7 02:47:46,353 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:47:46,353 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testBackupDatastore 02:47:46,354 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 02:47:46,354 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore to akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 02:47:46,354 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testBackupDatastore on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore on shard member-1-shard-cars-oper_testBackupDatastore whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-1100458479] 02:47:46,354 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testBackupDatastore 02:47:46,354 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testBackupDatastore, persistent : true 02:47:46,354 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-1100458479] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:46,355 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 02:47:46,355 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore to akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 02:47:46,355 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testBackupDatastore on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore on shard member-1-shard-people-oper_testBackupDatastore whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-727740952] 02:47:46,355 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/shard-member-1:snapshot-read#-1871190096] 02:47:46,355 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#1146067970 created and ready for shard:member-1-shard-cars-oper_testBackupDatastore 02:47:46,355 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testBackupDatastore with journal batch size 1 02:47:46,356 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-people-oper_testBackupDatastore, persistent : true 02:47:46,357 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:46,357 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:46,357 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 02:47:46,357 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testBackupDatastore is no longer the leader 02:47:46,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 02:47:46,358 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#-938030106 created and ready for shard:member-1-shard-people-oper_testBackupDatastore 02:47:46,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 02:47:46,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testBackupDatastore 02:47:46,358 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from null to Follower 02:47:46,358 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Starting new election term 1 02:47:46,358 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:47:46,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 02:47:46,358 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/shard-member-1:snapshot-read#-2017950674] 02:47:46,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testBackupDatastore is no longer the leader 02:47:46,359 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 02:47:46,359 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testBackupDatastore is initialized 02:47:46,359 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Follower to Candidate 02:47:46,359 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#1146067970] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:46,359 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Received ElectionTimeout 02:47:46,359 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Leader): Election: Leader has following peers: [] 02:47:46,359 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:47:46,359 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 02:47:46,359 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: became leader with frontend state for [] 02:47:46,360 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Candidate to Leader 02:47:46,360 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testBackupDatastore with journal batch size 1 02:47:46,360 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , registered listener akka://cluster-test/user/shardmanager-oper_testBackupDatastore 02:47:46,361 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testBackupDatastore, leaderId=member-1-shard-cars-oper_testBackupDatastore, leaderPayloadVersion=13] 02:47:46,361 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testBackupDatastore from Candidate to Leader 02:47:46,361 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:46,361 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:46,362 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 02:47:46,362 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testBackupDatastore is no longer the leader 02:47:46,362 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from null to Follower 02:47:46,362 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 02:47:46,362 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testBackupDatastore is initialized 02:47:46,362 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 02:47:46,362 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 02:47:46,362 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-people-oper_testBackupDatastore 02:47:46,363 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Starting new election term 1 02:47:46,363 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:47:46,363 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 02:47:46,363 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testBackupDatastore is no longer the leader 02:47:46,363 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Follower to Candidate 02:47:46,364 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Received ElectionTimeout 02:47:46,364 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Leader): Election: Leader has following peers: [] 02:47:46,364 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:47:46,364 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 02:47:46,365 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: became leader with frontend state for [] 02:47:46,365 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Candidate to Leader 02:47:46,365 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , registered listener akka://cluster-test/user/shardmanager-oper_testBackupDatastore 02:47:46,365 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testBackupDatastore, leaderId=member-1-shard-people-oper_testBackupDatastore, leaderPayloadVersion=13] 02:47:46,365 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testBackupDatastore from Candidate to Leader 02:47:46,365 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testBackupDatastore is ready 02:47:46,401 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore#-70886442] 02:47:46,402 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 02:47:46,402 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-1100458479] 02:47:46,453 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore#-70886442] 02:47:46,453 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 02:47:46,454 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-727740952] 02:47:46,474 PM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 02:47:46,477 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 02:47:46,478 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 02:47:46,484 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onGetSnapshot 02:47:46,484 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 02:47:46,485 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:47:46,486 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onGetSnapshot 02:47:46,486 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:47:46,489 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 02:47:46,489 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onGetSnapshot 02:47:46,489 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:47:46,491 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onGetSnapshot 02:47:46,491 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:47:46,510 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:47:46,512 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:47:46,513 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:47:46,521 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:47:46,514 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@117816d, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 02:47:46,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-cars-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@55f8014c, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testBackupDatastore, ServerConfigPayload=null]] 02:47:46,527 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@ffeae38, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 02:47:46,527 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@374b1ccf, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 02:47:46,528 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 02:47:46,528 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: All shard snapshots received 02:47:46,577 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ClusterAdminRpcService - Successfully backed up datastore to file target/testBackupDatastore 02:47:46,611 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#-800981009] 02:47:46,611 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Received unhandled message org.opendaylight.controller.cluster.datastore.DatastoreContext@61ab351e 02:47:46,613 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:47:46,613 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-24374518] 02:47:46,649 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testBackupDatastore 02:47:46,649 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: Aborting 0 pending queued transactions 02:47:46,655 PM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 02:47:46,658 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 02:47:46,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 02:47:46,659 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 02:47:46,659 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 02:47:46,660 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onGetSnapshot 02:47:46,660 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:47:46,660 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:47:46,661 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onGetSnapshot 02:47:46,661 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:47:46,661 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3d915164, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 02:47:46,662 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:47:46,662 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3356b8a6, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 02:47:46,661 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onGetSnapshot 02:47:46,662 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:47:46,663 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:47:46,663 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7ff41255, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 02:47:46,663 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 02:47:46,873 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond 02:47:46,876 PM [cluster-test-akka.actor.default-dispatcher-23] [ERROR] ClusterAdminRpcService - Failed to back up datastore to file target/testBackupDatastore java.util.concurrent.TimeoutException: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerGetSnapshotReplyActor.onReceive(ShardManagerGetSnapshotReplyActor.java:66) at akka.actor.UntypedAbstractActor$$anonfun$receive$1.applyOrElse(AbstractActor.scala:332) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 02:47:46,888 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 02:47:46,890 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testBackupDatastore 02:47:46,892 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testBackupDatastore 02:47:46,892 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 02:47:46,895 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onShutDown 02:47:46,897 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@207016d4 02:47:46,897 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-config_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@207016d4 02:47:46,899 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-people-config_testBackupDatastore 02:47:46,899 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Aborting 0 pending queued transactions 02:47:46,900 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:47:46,900 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:47:46,905 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 02:47:46,906 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testBackupDatastore 02:47:46,906 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onShutDown 02:47:46,906 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@12e0b18c 02:47:46,906 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@12e0b18c 02:47:46,907 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testBackupDatastore 02:47:46,907 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Aborting 0 pending queued transactions 02:47:46,907 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testBackupDatastore 02:47:46,907 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 02:47:46,908 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onShutDown 02:47:46,908 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5702cd36 02:47:46,908 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5702cd36 02:47:46,908 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testBackupDatastore 02:47:46,908 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Aborting 0 pending queued transactions 02:47:46,909 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:47:46,909 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:47:46,916 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:47:46,918 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:47:46,940 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:47:46,998 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:47:47,34 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4750156972256722630] 02:47:47,35 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:47:47,40 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:47:47,40 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:47:47,42 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:47:47,42 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:47:47,65 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:47,67 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:47:47,67 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:47:47,67 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:47:47,72 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:47,76 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 02:47:47,85 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-805048711] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:47,89 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:47:47,95 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:47:47,95 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddReplicasForAllShards 02:47:47,96 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testAddReplicasForAllShards 02:47:47,96 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:47:47,96 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 02:47:47,97 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 02:47:47,98 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 02:47:47,99 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@56d35f3b 02:47:47,100 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 02:47:47,101 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:47:47,101 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddReplicasForAllShards 02:47:47,102 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 02:47:47,102 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 02:47:47,102 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards on shard member-1-shard-cars-config_testAddReplicasForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#306984047] 02:47:47,103 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testAddReplicasForAllShards 02:47:47,104 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 02:47:47,104 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 02:47:47,104 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards on shard member-1-shard-people-config_testAddReplicasForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1254968942] 02:47:47,105 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddReplicasForAllShards, persistent : true 02:47:47,105 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#306984047] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:47,106 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#-236577618] 02:47:47,106 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#665056263 created and ready for shard:member-1-shard-cars-config_testAddReplicasForAllShards 02:47:47,106 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1254968942] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:47,109 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-1-shard-people-config_testAddReplicasForAllShards, persistent : true 02:47:47,113 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddReplicasForAllShards with journal batch size 1 02:47:47,113 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#665056263] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:47,114 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#-236577618] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:47,115 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-1-shard-people-config_testAddReplicasForAllShards with journal batch size 1 02:47:47,116 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:47,116 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:47,117 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 02:47:47,117 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddReplicasForAllShards is initialized 02:47:47,118 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#-1907477384] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:47,118 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#1989581855] 02:47:47,119 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#1989581855] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:47,119 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:47,119 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddReplicasForAllShards is no longer the leader 02:47:47,119 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 02:47:47,119 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 02:47:47,119 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 02:47:47,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-config_testAddReplicasForAllShards 02:47:47,120 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 02:47:47,120 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:47:47,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:47,120 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddReplicasForAllShards is no longer the leader 02:47:47,120 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Follower to Candidate 02:47:47,122 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:47,123 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:47,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 02:47:47,123 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testAddReplicasForAllShards is initialized 02:47:47,123 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 02:47:47,123 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:47,123 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testAddReplicasForAllShards is no longer the leader 02:47:47,123 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 02:47:47,123 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:47:47,123 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 02:47:47,124 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: became leader with frontend state for [] 02:47:47,124 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 02:47:47,124 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Candidate to Leader 02:47:47,124 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 02:47:47,124 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 02:47:47,124 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-config_testAddReplicasForAllShards 02:47:47,124 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 02:47:47,125 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:47:47,125 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:47,125 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testAddReplicasForAllShards is no longer the leader 02:47:47,126 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 02:47:47,126 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 02:47:47,126 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:47:47,126 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 02:47:47,126 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: became leader with frontend state for [] 02:47:47,126 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 02:47:47,127 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Candidate to Leader 02:47:47,128 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#-1907477384 created and ready for shard:member-1-shard-people-config_testAddReplicasForAllShards 02:47:47,128 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 02:47:47,128 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Follower to Candidate 02:47:47,128 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Candidate to Leader 02:47:47,128 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 02:47:47,128 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 02:47:47,129 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Candidate to Leader 02:47:47,129 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 02:47:47,151 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-805048711] 02:47:47,151 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:47:47,152 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#306984047] 02:47:47,202 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-805048711] 02:47:47,202 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 02:47:47,203 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1254968942] 02:47:47,203 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:47,210 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:47,216 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 02:47:47,219 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:47:47,220 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-1460309692] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:47,223 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:47:47,223 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddReplicasForAllShards 02:47:47,223 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testAddReplicasForAllShards 02:47:47,224 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:47:47,224 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 02:47:47,224 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 02:47:47,226 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 02:47:47,227 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@55cf1f33 02:47:47,227 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 02:47:47,228 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:47:47,228 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddReplicasForAllShards 02:47:47,229 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 02:47:47,230 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 02:47:47,230 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards on shard member-1-shard-cars-oper_testAddReplicasForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-459086011] 02:47:47,230 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testAddReplicasForAllShards 02:47:47,230 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddReplicasForAllShards, persistent : true 02:47:47,230 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 02:47:47,230 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 02:47:47,230 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards on shard member-1-shard-people-oper_testAddReplicasForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#77213644] 02:47:47,231 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddReplicasForAllShards with journal batch size 1 02:47:47,232 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-459086011] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:47,232 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#462521946 created and ready for shard:member-1-shard-cars-oper_testAddReplicasForAllShards 02:47:47,233 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/shard-member-1:snapshot-read#-1411121691] 02:47:47,233 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-oper_testAddReplicasForAllShards, persistent : true 02:47:47,234 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:47,234 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:47,235 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:47,235 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 02:47:47,235 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 02:47:47,235 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 02:47:47,235 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testAddReplicasForAllShards 02:47:47,235 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 02:47:47,235 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:47:47,235 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:47,235 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 02:47:47,236 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testAddReplicasForAllShards with journal batch size 1 02:47:47,236 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 02:47:47,236 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 02:47:47,236 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/shard-member-1:snapshot-read#995050228] 02:47:47,238 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 02:47:47,238 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 02:47:47,238 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:47:47,238 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 02:47:47,238 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: became leader with frontend state for [] 02:47:47,238 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 02:47:47,239 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:47,239 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:47,239 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:47,239 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testAddReplicasForAllShards is no longer the leader 02:47:47,239 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 02:47:47,239 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 02:47:47,240 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-oper_testAddReplicasForAllShards 02:47:47,240 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 02:47:47,240 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:47:47,240 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:47,240 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testAddReplicasForAllShards is no longer the leader 02:47:47,241 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 02:47:47,241 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 02:47:47,241 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:47:47,241 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 02:47:47,241 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: became leader with frontend state for [] 02:47:47,232 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#77213644] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:47,236 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#-1664525813 created and ready for shard:member-1-shard-people-oper_testAddReplicasForAllShards 02:47:47,241 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 02:47:47,241 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 02:47:47,241 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 02:47:47,236 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 02:47:47,242 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddReplicasForAllShards is initialized 02:47:47,242 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 02:47:47,242 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testAddReplicasForAllShards is initialized 02:47:47,242 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 02:47:47,242 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 02:47:47,242 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 02:47:47,243 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Candidate to Leader 02:47:47,243 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 02:47:47,243 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Candidate to Leader 02:47:47,243 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 02:47:47,278 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-1460309692] 02:47:47,279 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 02:47:47,279 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-459086011] 02:47:47,330 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-1460309692] 02:47:47,330 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 02:47:47,331 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#77213644] 02:47:47,336 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@7747aa57] 02:47:47,341 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 02:47:47,344 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-pets-config_testAddReplicasForAllShards, persistent : true 02:47:47,345 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-pets-config_testAddReplicasForAllShards with journal batch size 1 02:47:47,345 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards-notifier#-1805607620 created and ready for shard:member-1-shard-pets-config_testAddReplicasForAllShards 02:47:47,346 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#-1123287569] 02:47:47,348 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:47,348 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:47,348 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 02:47:47,349 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testAddReplicasForAllShards is initialized 02:47:47,349 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:47,349 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testAddReplicasForAllShards is no longer the leader 02:47:47,349 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 02:47:47,349 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 02:47:47,349 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 02:47:47,349 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-pets-config_testAddReplicasForAllShards 02:47:47,349 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 02:47:47,349 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:47:47,350 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:47,350 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testAddReplicasForAllShards is no longer the leader 02:47:47,350 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Follower to Candidate 02:47:47,350 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 02:47:47,351 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 02:47:47,351 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:47:47,351 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 02:47:47,351 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: became leader with frontend state for [] 02:47:47,351 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Candidate to Leader 02:47:47,351 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 02:47:47,351 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 02:47:47,351 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testAddReplicasForAllShards from Candidate to Leader 02:47:47,351 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 02:47:47,397 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-805048711] 02:47:47,398 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 02:47:47,398 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#1129676116] 02:47:47,417 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 02:47:47,444 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4015434140233191685] 02:47:47,445 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:47:47,451 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:47:47,451 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:47:47,458 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:47:47,458 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:47:47,496 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:47,508 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:47,511 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 02:47:47,529 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-2133060443] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:47,529 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 02:47:47,538 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 02:47:47,544 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 02:47:47,548 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@119c7acf 02:47:47,548 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 02:47:47,549 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:47:47,549 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:47,552 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:47,555 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 02:47:47,556 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#539206904] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:47,557 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 02:47:47,560 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 02:47:47,561 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 02:47:47,562 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1b6411bf 02:47:47,562 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 02:47:47,564 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:47:47,847 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:47:47,898 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:47:47,901 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:47:47,901 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 02:47:47,902 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:47:47,902 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 02:47:48,60 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:47:48,61 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:47:48,61 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards 02:47:48,61 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 02:47:48,61 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards 02:47:48,61 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 02:47:48,61 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 02:47:48,66 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:47:48,66 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-pets-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards 02:47:48,66 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-pets-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 02:47:48,66 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards 02:47:48,66 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 02:47:48,67 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards 02:47:48,67 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 02:47:48,67 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 02:47:48,73 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:47:48,73 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:47:48,73 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 02:47:48,73 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 02:47:48,80 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@11a73737] 02:47:48,83 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 02:47:48,85 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-1499946017] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:48,85 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-2-shard-pets-config_testAddReplicasForAllShards, persistent : true 02:47:48,86 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#-752394586 created and ready for shard:member-2-shard-pets-config_testAddReplicasForAllShards 02:47:48,86 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#-752394586] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:48,86 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#-1412274425] 02:47:48,87 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#-1412274425] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:48,87 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-2-shard-pets-config_testAddReplicasForAllShards with journal batch size 1 02:47:48,87 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=no-leader-ns, moduleName=no-leader-module, shardName=no-leader, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@e15e90c] 02:47:48,88 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2 doCreateShard: shardId: member-2-shard-no-leader-oper_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 02:47:48,89 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:48,89 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:48,89 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-no-leader-oper_testAddReplicasForAllShards, persistent : true 02:47:48,90 PM [main] [INFO] ClusterAdminRpcService - Adding replicas for all shards 02:47:48,90 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-no-leader-oper_testAddReplicasForAllShards with journal batch size 1 02:47:48,90 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testAddReplicasForAllShards 02:47:48,90 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 02:47:48,90 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testAddReplicasForAllShards is initialized 02:47:48,91 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards#1231029198] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:48,91 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, no-leader] for data store oper_testAddReplicasForAllShards 02:47:48,93 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 02:47:48,96 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 02:47:48,96 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=no-leader] 02:47:48,97 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:47:48,98 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 02:47:48,98 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#-507785518 created and ready for shard:member-2-shard-no-leader-oper_testAddReplicasForAllShards 02:47:48,98 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 02:47:48,98 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for people forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 02:47:48,98 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=no-leader, waitUntilReady=true] 02:47:48,99 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for no-leader forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 02:47:48,102 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#-507785518] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:48,105 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:48,105 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-pets-config_testAddReplicasForAllShards is no longer the leader 02:47:48,105 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 02:47:48,109 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 02:47:48,108 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 02:47:48,110 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 02:47:48,110 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=pets] 02:47:48,110 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:47:48,110 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 02:47:48,110 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 02:47:48,110 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for people forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 02:47:48,111 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 02:47:48,111 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for pets forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 02:47:48,111 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-459086011, primaryVersion=13] 02:47:48,113 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 02:47:48,113 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#77213644, primaryVersion=13] 02:47:48,113 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=no-leader, waitUntilReady=true] 02:47:48,113 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: No shard found for no-leader 02:47:48,105 PM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 02:47:48,114 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 02:47:48,114 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testAddReplicasForAllShards from null to Follower 02:47:48,117 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#146942928] 02:47:48,118 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:48,118 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:48,118 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [no-leader] 02:47:48,118 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards is initialized 02:47:48,118 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:48,118 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-no-leader-oper_testAddReplicasForAllShards is no longer the leader 02:47:48,119 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 02:47:48,119 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#306984047, primaryVersion=13] 02:47:48,119 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 02:47:48,119 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , received role change from null to Follower 02:47:48,119 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1254968942, primaryVersion=13] 02:47:48,119 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=pets, waitUntilReady=true] 02:47:48,119 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for pets: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#1129676116, primaryVersion=13] 02:47:48,120 PM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 02:47:48,120 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 02:47:48,120 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-no-leader-oper_testAddReplicasForAllShards from null to Follower 02:47:48,124 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#146942928] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:48,127 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Failed to find leader for shard no-leader: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1051) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:207) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 02:47:48,127 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] ClusterAdminRpcService - Failed to add replica for shard no-leader, type Operational org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1051) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:207) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 02:47:48,129 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-459086011 for shard member-2-shard-cars-oper_testAddReplicasForAllShards 02:47:48,130 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 1 02:47:48,132 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1254968942 for shard member-2-shard-people-config_testAddReplicasForAllShards 02:47:48,133 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-2-shard-people-config_testAddReplicasForAllShards, persistent : true 02:47:48,133 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#485477918] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:48,134 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#306984047 for shard member-2-shard-cars-config_testAddReplicasForAllShards 02:47:48,134 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#1129676116 for shard member-2-shard-pets-config_testAddReplicasForAllShards 02:47:48,134 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-notifier#-957260566 created and ready for shard:member-2-shard-people-config_testAddReplicasForAllShards 02:47:48,134 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#-471366959] 02:47:48,135 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-notifier#-957260566] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:48,135 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-2-shard-cars-config_testAddReplicasForAllShards, persistent : true 02:47:48,135 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-cars-oper_testAddReplicasForAllShards, persistent : true 02:47:48,136 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#1868294331 created and ready for shard:member-2-shard-cars-config_testAddReplicasForAllShards 02:47:48,134 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-2-shard-people-config_testAddReplicasForAllShards with journal batch size 1 02:47:48,136 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#1332754479] 02:47:48,137 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#881300810 created and ready for shard:member-2-shard-cars-oper_testAddReplicasForAllShards 02:47:48,137 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#-1380942399] 02:47:48,137 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testAddReplicasForAllShards with journal batch size 1 02:47:48,137 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testAddReplicasForAllShards with journal batch size 1 02:47:48,137 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:48,138 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:48,138 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 02:47:48,138 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:48,138 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testAddReplicasForAllShards is no longer the leader 02:47:48,138 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testAddReplicasForAllShards is initialized 02:47:48,138 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 02:47:48,138 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:48,138 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 02:47:48,138 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:48,138 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:48,139 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:48,139 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testAddReplicasForAllShards from null to Follower 02:47:48,139 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:48,139 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddReplicasForAllShards is no longer the leader 02:47:48,139 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 02:47:48,139 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddReplicasForAllShards is initialized 02:47:48,139 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:48,139 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 02:47:48,139 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 02:47:48,139 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 02:47:48,139 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 02:47:48,139 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddReplicasForAllShards from null to Follower 02:47:48,140 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards, votingMember=true], state: Idle 02:47:48,143 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-2-shard-people-oper_testAddReplicasForAllShards, persistent : true 02:47:48,144 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#-2133071976] 02:47:48,144 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-notifier#-778504443 created and ready for shard:member-2-shard-people-oper_testAddReplicasForAllShards 02:47:48,144 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testAddReplicasForAllShards with journal batch size 1 02:47:48,144 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards, votingMember=true], state: Idle 02:47:48,144 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards, votingMember=true], state: Idle 02:47:48,145 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards, votingMember=true], state: Idle 02:47:48,145 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#77213644 for shard member-2-shard-people-oper_testAddReplicasForAllShards 02:47:48,146 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards, votingMember=true] 02:47:48,146 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:48,147 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:47:48,147 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:47:48,147 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testAddReplicasForAllShards is no longer the leader 02:47:48,147 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:47:48,147 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 02:47:48,148 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddReplicasForAllShards is initialized 02:47:48,148 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 02:47:48,148 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testAddReplicasForAllShards is initialized 02:47:48,148 PM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards, votingMember=true], state: Idle 02:47:48,148 PM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards, votingMember=true] 02:47:48,148 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:47:48,148 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards, votingMember=true] 02:47:48,148 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:47:48,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards, votingMember=true] 02:47:48,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:47:48,149 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 02:47:48,149 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 02:47:48,149 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 02:47:48,149 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddReplicasForAllShards from null to Follower 02:47:48,149 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards, votingMember=true] 02:47:48,153 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testAddReplicasForAllShards from null to Follower 02:47:48,153 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:47:48,150 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddReplicasForAllShards 02:47:48,154 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 02:47:48,154 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:47:48,149 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-config_testAddReplicasForAllShards 02:47:48,155 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 02:47:48,155 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:47:48,153 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddReplicasForAllShards 02:47:48,155 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 02:47:48,155 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:47:48,155 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-cars-config_testAddReplicasForAllShards 02:47:48,156 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 02:47:48,149 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-pets-config_testAddReplicasForAllShards 02:47:48,150 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-oper_testAddReplicasForAllShards 02:47:48,156 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5b978d84, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddReplicasForAllShards, ServerConfigPayload=null] 02:47:48,165 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 02:47:48,165 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 02:47:48,167 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:47:48,167 PM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-people-oper_testAddReplicasForAllShards 02:47:48,165 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 02:47:48,168 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:47:48,168 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-pets-config_testAddReplicasForAllShards 02:47:48,160 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-cars-oper_testAddReplicasForAllShards 02:47:48,168 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 02:47:48,168 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 02:47:48,160 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-people-config_testAddReplicasForAllShards 02:47:48,168 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@43acd3c5, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testAddReplicasForAllShards, ServerConfigPayload=null] 02:47:48,168 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 02:47:48,168 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 02:47:48,168 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@a3e1359, electionTerm=1, electionVotedFor=member-1-shard-people-config_testAddReplicasForAllShards, ServerConfigPayload=null] 02:47:48,168 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 02:47:48,168 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@69d87821, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddReplicasForAllShards, ServerConfigPayload=null] 02:47:48,168 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 02:47:48,170 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): sendInstallSnapshot 02:47:48,170 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 02:47:48,170 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-people-config_testAddReplicasForAllShards: 229 02:47:48,168 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 02:47:48,170 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@281303e9, electionTerm=1, electionVotedFor=member-1-shard-pets-config_testAddReplicasForAllShards, ServerConfigPayload=null] 02:47:48,171 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 02:47:48,171 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 02:47:48,171 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-pets-config_testAddReplicasForAllShards: 229 02:47:48,170 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-people-oper_testAddReplicasForAllShards: 229 02:47:48,171 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 02:47:48,171 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testAddReplicasForAllShards: 229 02:47:48,170 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): sendInstallSnapshot 02:47:48,171 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testAddReplicasForAllShards: 229 02:47:48,172 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddReplicasForAllShards, member-2-shard-cars-oper_testAddReplicasForAllShards), Chunk: 1/1 02:47:48,172 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:47:48,172 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 02:47:48,173 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: SnapshotComplete received 02:47:48,173 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: handled a snapshot response 02:47:48,175 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddReplicasForAllShards, member-2-shard-people-oper_testAddReplicasForAllShards), Chunk: 1/1 02:47:48,176 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:47:48,176 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 02:47:48,176 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: SnapshotComplete received 02:47:48,176 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: handled a journal response 02:47:48,176 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: handled a snapshot response 02:47:48,176 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: handled a journal response 02:47:48,176 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-people-config_testAddReplicasForAllShards), Chunk: 1/1 02:47:48,176 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:47:48,176 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 02:47:48,176 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: SnapshotComplete received 02:47:48,177 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: handled a snapshot response 02:47:48,177 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: handled a journal response 02:47:48,179 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-pets-config_testAddReplicasForAllShards), Chunk: 1/1 02:47:48,180 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:47:48,180 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 02:47:48,180 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: SnapshotComplete received 02:47:48,180 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: handled a journal response 02:47:48,180 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: handled a snapshot response 02:47:48,187 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-cars-config_testAddReplicasForAllShards), Chunk: 1/1 02:47:48,187 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:47:48,187 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 02:47:48,187 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: SnapshotComplete received 02:47:48,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: handled a snapshot response 02:47:48,186 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 02:47:48,183 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 02:47:48,190 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 02:47:48,184 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 02:47:48,190 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 02:47:48,190 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 02:47:48,190 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 02:47:48,184 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 02:47:48,190 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 02:47:48,190 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: handled a journal response 02:47:48,190 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 02:47:48,181 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 02:47:48,190 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 02:47:48,190 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 02:47:48,190 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 02:47:48,190 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 02:47:48,194 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 02:47:48,194 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 02:47:48,194 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 02:47:48,194 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-people-oper_testAddReplicasForAllShards 02:47:48,194 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-pets-config_testAddReplicasForAllShards 02:47:48,194 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-people-config_testAddReplicasForAllShards 02:47:48,194 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Converted InstallSnapshot from leader: member-1-shard-people-oper_testAddReplicasForAllShards to state 02:47:48,195 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Converted InstallSnapshot from leader: member-1-shard-pets-config_testAddReplicasForAllShards to state 02:47:48,194 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Converted InstallSnapshot from leader: member-1-shard-people-config_testAddReplicasForAllShards to state 02:47:48,195 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 02:47:48,195 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testAddReplicasForAllShards 02:47:48,196 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-oper_testAddReplicasForAllShards to state 02:47:48,196 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 02:47:48,196 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-cars-config_testAddReplicasForAllShards 02:47:48,196 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-pets-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 02:47:48,196 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-config_testAddReplicasForAllShards to state 02:47:48,196 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 02:47:48,197 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@54a05eef, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 02:47:48,197 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 02:47:48,196 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 02:47:48,198 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 02:47:48,198 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 02:47:48,197 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 02:47:48,198 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 02:47:48,198 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5315d73b, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 02:47:48,198 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 02:47:48,198 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 02:47:48,198 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 02:47:48,199 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:47:48,199 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 02:47:48,199 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:47:48,199 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 02:47:48,199 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 02:47:48,199 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 02:47:48,199 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 02:47:48,199 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 02:47:48,199 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot 02:47:48,199 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot 02:47:48,199 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-pets-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 02:47:48,200 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-2-shard-people-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 02:47:48,200 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@134ad929, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 02:47:48,200 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 02:47:48,200 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:47:48,200 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 02:47:48,200 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 02:47:48,200 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 02:47:48,200 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot 02:47:48,200 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 02:47:48,201 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 02:47:48,200 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 02:47:48,201 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=-1] 02:47:48,201 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5062ef03, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 02:47:48,201 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 02:47:48,202 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:47:48,202 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 02:47:48,202 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Moving last applied index from -1 to -1 02:47:48,202 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 02:47:48,202 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot 02:47:48,202 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 02:47:48,202 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 02:47:48,203 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 02:47:48,203 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@4893102d, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 02:47:48,203 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=-1] 02:47:48,203 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 02:47:48,203 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:47:48,203 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 02:47:48,203 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 02:47:48,203 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Moving last applied index from -1 to -1 02:47:48,203 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 02:47:48,204 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot 02:47:48,204 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-people-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 02:47:48,228 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-config_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#-410623895] with name member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher 02:47:48,229 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#-216645825] with name member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher 02:47:48,229 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-people-config_testAddReplicasForAllShards: state snapshot applied in 29.78 ms 02:47:48,229 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddReplicasForAllShards: state snapshot applied in 29.00 ms 02:47:48,230 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Done applying snapshot 02:47:48,230 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-oper_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#1504800998] with name member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher 02:47:48,231 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#-216645825] 02:47:48,231 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#-410623895] 02:47:48,231 PM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#1504800998] 02:47:48,230 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Done applying snapshot 02:47:48,231 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#-1353098111] with name member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher 02:47:48,231 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddReplicasForAllShards: state snapshot applied in 29.54 ms 02:47:48,232 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Done applying snapshot 02:47:48,231 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#-1353098111] 02:47:48,232 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddReplicasForAllShards: publishChanges: rootPath: / 02:47:48,232 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 02:47:48,231 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-oper_testAddReplicasForAllShards: state snapshot applied in 27.06 ms 02:47:48,232 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-config_testAddReplicasForAllShards: publishChanges: rootPath: / 02:47:48,232 PM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 02:47:48,230 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 02:47:48,236 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: SnapshotComplete received 02:47:48,238 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: handled a journal response 02:47:48,238 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: handled a snapshot response 02:47:48,239 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#-1676586167] 02:47:48,240 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-pets-config_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#-1676586167] with name member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher 02:47:48,240 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-pets-config_testAddReplicasForAllShards: publishChanges: rootPath: / 02:47:48,240 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-pets-config_testAddReplicasForAllShards: state snapshot applied in 40.24 ms 02:47:48,240 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Done applying snapshot 02:47:48,240 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 02:47:48,240 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: SnapshotComplete received 02:47:48,241 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: handled a snapshot response 02:47:48,241 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: handled a journal response 02:47:48,231 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-people-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 02:47:48,234 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Done applying snapshot 02:47:48,248 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 02:47:48,248 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: SnapshotComplete received 02:47:48,246 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher: 13.96 ms 02:47:48,248 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: handled a snapshot response 02:47:48,249 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 02:47:48,249 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 02:47:48,250 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 02:47:48,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: SnapshotComplete received 02:47:48,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: handled a snapshot response 02:47:48,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: handled a journal response 02:47:48,250 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-people-oper_testAddReplicasForAllShards] 02:47:48,232 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 02:47:48,252 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 02:47:48,252 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 02:47:48,252 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 02:47:48,252 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testAddReplicasForAllShards] 02:47:48,251 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]] 02:47:48,253 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: SnapshotComplete received 02:47:48,247 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-people-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 02:47:48,246 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 02:47:48,253 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 02:47:48,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 02:47:48,253 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 02:47:48,244 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher: 12.05 ms 02:47:48,253 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-people-config_testAddReplicasForAllShards] 02:47:48,244 PM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher: 12.05 ms 02:47:48,244 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher: 11.96 ms 02:47:48,244 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-pets-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher: 4.335 ms 02:47:48,253 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]] 02:47:48,253 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 02:47:48,253 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 02:47:48,253 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testAddReplicasForAllShards] 02:47:48,253 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]] 02:47:48,253 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-pets-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 02:47:48,254 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 02:47:48,254 PM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-pets-config_testAddReplicasForAllShards] 02:47:48,254 PM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]] 02:47:48,253 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: handled a snapshot response 02:47:48,252 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]] 02:47:48,249 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: handled a journal response 02:47:48,254 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: handled a journal response 02:47:48,255 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,255 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,258 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,259 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,256 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,259 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,261 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,261 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,261 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=83642352-56c4-4cde-a072-8a3c31f0693a}, logIndex: 0, isSendImmediate: true 02:47:48,261 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=cc4646ba-b448-4e59-86d6-c851fc4ed912}, logIndex: 0, isSendImmediate: true 02:47:48,255 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,261 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,261 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=fba840c9-3c81-4b93-b69e-1bec3fb01862}, logIndex: 0, isSendImmediate: true 02:47:48,262 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:48,262 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=1d612069-35ae-4f65-b8dc-97b3e489e232}, logIndex: 0, isSendImmediate: true 02:47:48,262 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:48,262 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:48,262 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testAddReplicasForAllShards 02:47:48,262 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:48,262 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testAddReplicasForAllShards 02:47:48,262 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testAddReplicasForAllShards 02:47:48,262 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=9b2dff06-f74e-4038-83d0-1f4139a97484}, logIndex: 0, isSendImmediate: true 02:47:48,263 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:48,262 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testAddReplicasForAllShards 02:47:48,263 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-pets-config_testAddReplicasForAllShards 02:47:48,262 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]]]] 02:47:48,263 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-config_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-people-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]]]] 02:47:48,270 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards, votingMember=true] 02:47:48,273 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards, votingMember=true] 02:47:48,263 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-pets-config_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]]]] 02:47:48,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,274 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards, votingMember=true] 02:47:48,262 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 02:47:48,275 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,275 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards, votingMember=true] 02:47:48,276 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,263 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 02:47:48,277 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards, votingMember=true] 02:47:48,278 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,283 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]]]] 02:47:48,284 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 02:47:48,284 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:47:48,284 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] 02:47:48,284 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,284 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testAddReplicasForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards, votingState=VOTING]] 02:47:48,285 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Log size is now 1 02:47:48,287 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-config_testAddReplicasForAllShards} for shard people from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1254968942 02:47:48,287 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard people 02:47:48,288 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-config_testAddReplicasForAllShards 02:47:48,288 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [people] 02:47:48,288 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]]]] 02:47:48,288 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 02:47:48,288 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:47:48,288 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] 02:47:48,288 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,289 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-pets-config_testAddReplicasForAllShards, address=null, votingState=VOTING]] 02:47:48,289 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Log size is now 1 02:47:48,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 02:47:48,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 02:47:48,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:47:48,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] 02:47:48,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-oper_testAddReplicasForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards, votingState=VOTING]] 02:47:48,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Log size is now 1 02:47:48,291 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]]]] 02:47:48,291 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 02:47:48,291 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:47:48,291 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] 02:47:48,291 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,291 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testAddReplicasForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards, votingState=VOTING]] 02:47:48,291 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Log size is now 1 02:47:48,290 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:48,285 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:48,289 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddReplicasForAllShards} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-459086011 02:47:48,293 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 02:47:48,293 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddReplicasForAllShards 02:47:48,293 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 02:47:48,293 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-oper_testAddReplicasForAllShards} for shard people from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#77213644 02:47:48,293 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard people 02:47:48,293 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-oper_testAddReplicasForAllShards 02:47:48,294 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars, people] 02:47:48,294 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:47:48,289 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 02:47:48,294 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 2 02:47:48,289 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:47:48,294 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddReplicasForAllShards} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#306984047 02:47:48,294 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 02:47:48,294 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddReplicasForAllShards 02:47:48,294 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars, people] 02:47:48,294 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-pets-config_testAddReplicasForAllShards} for shard pets from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#1129676116 02:47:48,294 PM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 02:47:48,295 PM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 3 02:47:48,295 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 02:47:48,295 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 4 02:47:48,295 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 02:47:48,295 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 5 02:47:48,294 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard pets 02:47:48,295 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-pets-config_testAddReplicasForAllShards 02:47:48,295 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, cars, people] 02:47:48,295 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 02:47:48,295 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 02:47:48,295 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 02:47:48,295 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:47:48,295 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:47:48,295 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:47:48,295 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 02:47:48,295 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClusterAdminRpcService - onSuccess for shard pets, type Config 02:47:48,296 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 6 02:47:48,296 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 02:47:48,296 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 02:47:48,297 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 02:47:48,289 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 02:47:48,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 02:47:48,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:47:48,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] 02:47:48,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testAddReplicasForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards, votingState=VOTING]] 02:47:48,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Log size is now 1 02:47:48,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:48,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 02:47:48,297 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 02:47:48,297 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-2133060443] 02:47:48,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-pets-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 02:47:48,298 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:47:48,299 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#835545562] 02:47:48,300 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,300 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#835545562] 02:47:48,300 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 02:47:48,300 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-pets-config_testAddReplicasForAllShards 02:47:48,300 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 02:47:48,300 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-pets-config_testAddReplicasForAllShards is no longer the leader 02:47:48,300 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done true 02:47:48,300 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,300 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 02:47:48,300 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddReplicasForAllShards 02:47:48,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 02:47:48,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 02:47:48,301 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done true 02:47:48,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:48,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 02:47:48,301 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 02:47:48,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 02:47:48,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-people-oper_testAddReplicasForAllShards 02:47:48,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 02:47:48,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testAddReplicasForAllShards is no longer the leader 02:47:48,303 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done true 02:47:48,303 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 02:47:48,291 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-people-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:48,304 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 02:47:48,305 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 02:47:48,305 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 02:47:48,305 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 02:47:48,305 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 02:47:48,305 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,305 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 02:47:48,305 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-people-config_testAddReplicasForAllShards 02:47:48,305 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 02:47:48,305 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testAddReplicasForAllShards is no longer the leader 02:47:48,305 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done true 02:47:48,306 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 02:47:48,307 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 02:47:48,307 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:47:48,307 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 02:47:48,307 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 02:47:48,307 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] 02:47:48,307 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 02:47:48,308 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:47:48,308 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 02:47:48,308 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 02:47:48,308 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:48,308 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] 02:47:48,308 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:48,308 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 02:47:48,309 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 02:47:48,310 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 02:47:48,310 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 02:47:48,312 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-2133060443] 02:47:48,312 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 02:47:48,312 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#485477918] 02:47:48,313 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#485477918] 02:47:48,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Commit index set to 0 02:47:48,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:47:48,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Setting last applied to 0 02:47:48,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 02:47:48,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] 02:47:48,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 02:47:48,314 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-2133060443] 02:47:48,315 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 02:47:48,315 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-1499946017] 02:47:48,315 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-1499946017] 02:47:48,315 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 02:47:48,315 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:47:48,315 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Setting last applied to 0 02:47:48,315 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 02:47:48,315 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#539206904] 02:47:48,315 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] 02:47:48,316 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 02:47:48,316 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 02:47:48,316 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:48,316 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-1314623215] 02:47:48,316 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 02:47:48,316 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 02:47:48,316 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 02:47:48,316 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:47:48,316 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Setting last applied to 0 02:47:48,316 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 02:47:48,316 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] 02:47:48,316 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-1314623215] 02:47:48,316 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:48,317 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 02:47:48,317 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 02:47:48,317 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#539206904] 02:47:48,317 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 02:47:48,317 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#-699138809] 02:47:48,318 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#-699138809] 02:47:48,318 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 02:47:48,318 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testAddReplicasForAllShards 02:47:48,319 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddReplicasForAllShards 02:47:48,319 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testAddReplicasForAllShards 02:47:48,319 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 02:47:48,319 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onShutDown 02:47:48,319 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onShutDown 02:47:48,320 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Commit index set to 0 02:47:48,320 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:47:48,320 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Setting last applied to 0 02:47:48,320 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 02:47:48,320 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] 02:47:48,320 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 02:47:48,320 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 02:47:48,320 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Commit index set to 0 02:47:48,321 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-2-shard-people-config_testAddReplicasForAllShards 02:47:48,321 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 02:47:48,321 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:47:48,321 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Setting last applied to 0 02:47:48,321 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 02:47:48,321 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] 02:47:48,321 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 02:47:48,321 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-pets-config_testAddReplicasForAllShards 02:47:48,322 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 02:47:48,322 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 02:47:48,322 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:47:48,322 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 02:47:48,322 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 02:47:48,322 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] 02:47:48,322 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:48,323 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 02:47:48,323 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 02:47:48,324 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 02:47:48,326 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Commit index set to 0 02:47:48,326 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:47:48,326 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Setting last applied to 0 02:47:48,326 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 02:47:48,326 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] 02:47:48,327 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onShutDown 02:47:48,327 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 02:47:48,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 02:47:48,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddReplicasForAllShards 02:47:48,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 02:47:48,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddReplicasForAllShards is no longer the leader 02:47:48,330 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done true 02:47:48,330 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testAddReplicasForAllShards 02:47:48,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 02:47:48,331 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 02:47:48,331 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 02:47:48,332 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 02:47:48,332 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddReplicasForAllShards 02:47:48,332 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testAddReplicasForAllShards 02:47:48,332 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 02:47:48,332 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 3 shards 02:47:48,332 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onShutDown 02:47:48,333 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onShutDown 02:47:48,333 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testAddReplicasForAllShards 02:47:48,333 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 02:47:48,333 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onShutDown 02:47:48,334 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 02:47:48,334 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testAddReplicasForAllShards 02:47:48,334 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 02:47:48,334 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 02:47:48,334 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 02:47:48,334 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 02:47:48,336 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:47:48,336 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:47:48,361 PM [cluster-test-akka.actor.default-dispatcher-34] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:47:48,363 PM [cluster-test-akka.actor.default-dispatcher-34] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:47:48,368 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:47:48,394 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 02:47:48,395 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testAddReplicasForAllShards 02:47:48,395 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddReplicasForAllShards 02:47:48,395 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testAddReplicasForAllShards 02:47:48,395 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 02:47:48,395 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onShutDown 02:47:48,395 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onShutDown 02:47:48,396 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onShutDown 02:47:48,396 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating leader transfer 02:47:48,396 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Initiating leader transfer 02:47:48,396 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating leader transfer 02:47:48,398 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 02:47:48,398 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 02:47:48,399 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5718d56e 02:47:48,399 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-pets-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5718d56e 02:47:48,399 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 02:47:48,400 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 02:47:48,400 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:48,401 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 02:47:48,401 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@357e33ed 02:47:48,401 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@357e33ed 02:47:48,402 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 02:47:48,402 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 02:47:48,402 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:48,399 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@194b5491 02:47:48,402 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@194b5491 02:47:48,402 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 02:47:48,402 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 02:47:48,402 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:49,367 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4015434140233191685] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:47:52,632 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:47:53,190 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:47:53,910 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:47:54,185 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2559, Up)]. 02:47:54,188 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:47:54,188 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:47:54,209 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:47:54,226 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:47:55,156 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:47:55,230 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:47:55,511 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:47:56,149 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:47:56,249 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:47:56,470 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:47:56,790 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:47:57,268 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:47:57,417 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer aborted 02:47:57,418 PM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.020 s 02:47:57,418 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In unpauseLeader 02:47:57,418 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 02:47:57,418 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 02:47:57,418 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 02:47:57,420 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-config_testAddReplicasForAllShards] 02:47:57,420 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 02:47:57,421 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:57,422 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 02:47:57,422 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 02:47:57,421 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-oper_testAddReplicasForAllShards] 02:47:57,422 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:57,423 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 02:47:57,423 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 02:47:57,423 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 02:47:57,423 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 02:47:57,423 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-1-shard-people-config_testAddReplicasForAllShards 02:47:57,423 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 02:47:57,427 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer aborted 02:47:57,427 PM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.030 s 02:47:57,428 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In unpauseLeader 02:47:57,428 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 02:47:57,428 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 02:47:57,428 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 02:47:57,428 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-oper_testAddReplicasForAllShards] 02:47:57,428 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer aborted 02:47:57,428 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:57,428 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.030 s 02:47:57,428 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In unpauseLeader 02:47:57,428 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 02:47:57,428 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 02:47:57,428 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 02:47:57,428 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 02:47:57,428 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 02:47:57,428 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 02:47:57,429 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 02:47:57,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testAddReplicasForAllShards] 02:47:57,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:57,429 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 02:47:57,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 02:47:57,429 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 02:47:57,429 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 02:47:57,430 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddReplicasForAllShards 02:47:57,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 02:47:57,430 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-pets-config_testAddReplicasForAllShards 02:47:57,430 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:47:57,430 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 02:47:57,431 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:47:57,431 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:47:57,431 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 02:47:57,431 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddReplicasForAllShards 02:47:57,432 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testAddReplicasForAllShards 02:47:57,432 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 02:47:57,432 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onShutDown 02:47:57,432 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddReplicasForAllShards 02:47:57,432 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onShutDown 02:47:57,432 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 02:47:57,433 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testAddReplicasForAllShards 02:47:57,433 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 02:47:57,433 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:47:57,433 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:47:57,434 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:47:57,434 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:47:57,438 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:47:57.441] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:47:57,481 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:47:57,503 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3248307750142836673] 02:47:57,504 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:47:57,507 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:47:57,508 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:47:57,509 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:47:57,509 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:47:57,529 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:47:57,529 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:47:57,529 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:47:57,531 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:57,534 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:57,537 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 02:47:57,547 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#937717968] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,549 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:47:57,567 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:47:57,567 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStates 02:47:57,567 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStates 02:47:57,568 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:47:57,568 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 02:47:57,568 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 02:47:57,575 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 02:47:57,577 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1f0c8888 02:47:57,577 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 02:47:57,578 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:57,578 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:47:57,578 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStates 02:47:57,580 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 02:47:57,580 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 02:47:57,580 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates on shard member-1-shard-cars-config_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:57,580 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStates 02:47:57,580 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:57,581 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStates, persistent : true 02:47:57,581 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,583 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 02:47:57,583 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 02:47:57,584 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStates, persistent : true 02:47:57,585 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStates with journal batch size 1 02:47:57,587 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#2030295261] 02:47:57,587 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#-376382700 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStates 02:47:57,587 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#1179783853] 02:47:57,587 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#-746231988 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStates 02:47:57,588 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:47:57,588 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 02:47:57,588 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 02:47:57,588 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates on shard member-1-shard-people-config_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#2007631992] 02:47:57,595 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#2007631992] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,595 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#338775429] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,595 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#-376382700] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,595 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#2030295261] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,595 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#-746231988] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,595 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#1179783853] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,601 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:47:57,601 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 02:47:57,601 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:47:57,601 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 02:47:57,601 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 02:47:57,602 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:47:57,602 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 02:47:57,602 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:47:57,602 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 02:47:57,602 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 02:47:57,602 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:57,602 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:47:57,602 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 02:47:57,602 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:57,602 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStates is initialized 02:47:57,602 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 02:47:57,603 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 02:47:57,603 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:47:57,603 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:57,603 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 02:47:57,603 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from null to Follower 02:47:57,603 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStates 02:47:57,603 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:57,603 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:47:57,604 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:57,604 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:47:57,604 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:57,604 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 02:47:57,604 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 02:47:57,604 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStates is initialized 02:47:57,604 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 02:47:57,604 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 02:47:57,604 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 02:47:57,604 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 02:47:57,604 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from null to Follower 02:47:57,605 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6721b34a 02:47:57,605 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 02:47:57,606 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:47:57,606 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:57,607 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:57,607 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:57,607 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-1-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1912470264] 02:47:57,607 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStates 02:47:57,608 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 02:47:57,608 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 02:47:57,608 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-1-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1254883418] 02:47:57,608 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStates, persistent : true 02:47:57,609 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1912470264] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,609 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#-476472708 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:57,609 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/shard-member-1:snapshot-read#2050561995] 02:47:57,610 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#-476472708] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,610 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStates, persistent : true 02:47:57,611 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#1096808984 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStates 02:47:57,611 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/shard-member-1:snapshot-read#-1418818382] 02:47:57,611 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 02:47:57,612 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 02:47:57,613 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:47:57,613 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 02:47:57,613 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:47:57,613 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 02:47:57,613 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 02:47:57,613 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:57,613 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:47:57,613 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 02:47:57,613 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStates is initialized 02:47:57,613 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:47:57,613 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 02:47:57,613 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:47:57,614 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 02:47:57,614 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 02:47:57,614 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:57,614 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:47:57,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 02:47:57,614 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:57,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStates is initialized 02:47:57,614 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 02:47:57,614 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 02:47:57,614 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 02:47:57,614 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from null to Follower 02:47:57,614 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:57,614 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 02:47:57,615 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 02:47:57,615 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 02:47:57,615 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from null to Follower 02:47:57,625 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:47:57,638 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8702847156426538529] 02:47:57,639 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:47:57,642 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:47:57,642 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:47:57,654 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:47:57,654 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:47:57,707 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:57,710 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:57,716 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 02:47:57,721 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 02:47:57,723 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1332099890] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,733 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 02:47:57,733 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStates 02:47:57,734 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStates 02:47:57,740 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 02:47:57,747 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1e63b442 02:47:57,747 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 02:47:57,748 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:47:57,748 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStates 02:47:57,748 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:57,749 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStates 02:47:57,750 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStates, persistent : true 02:47:57,751 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:57,751 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 02:47:57,751 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStates, persistent : true 02:47:57,752 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStates with journal batch size 1 02:47:57,753 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#2104155397] 02:47:57,753 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1721748110] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,753 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-682274936] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,753 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#1475382270] 02:47:57,753 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#1926111157 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStates 02:47:57,753 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#-1325454956] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,754 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 02:47:57,756 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 02:47:57,761 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#2104155397] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,761 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#1926111157] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,761 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#1475382270] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,761 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1192044054] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,763 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:47:57,763 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 02:47:57,763 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:47:57,763 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 02:47:57,763 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 02:47:57,763 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 02:47:57,763 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:57,764 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStates 02:47:57,764 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:57,764 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:47:57,764 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:47:57,764 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 02:47:57,764 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:47:57,764 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 02:47:57,764 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 02:47:57,764 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:57,764 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:47:57,765 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:57,765 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStates is no longer the leader 02:47:57,765 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 02:47:57,765 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 02:47:57,765 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStates is initialized 02:47:57,765 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 02:47:57,765 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStates is initialized 02:47:57,765 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#-1325454956 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStates 02:47:57,766 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 02:47:57,769 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@61b80c05 02:47:57,769 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 02:47:57,770 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:57,770 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStates is no longer the leader 02:47:57,770 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 02:47:57,770 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 02:47:57,770 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from null to Follower 02:47:57,770 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from null to Follower 02:47:57,771 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 02:47:57,772 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:47:57,772 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:57,773 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStates 02:47:57,774 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStates, persistent : true 02:47:57,774 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#369041399] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,775 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/shard-member-2:snapshot-read#-767197198] 02:47:57,775 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#1756462356 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:57,775 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 02:47:57,775 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#1756462356] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,776 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStates, persistent : true 02:47:57,776 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:47:57,776 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 02:47:57,776 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:47:57,776 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 02:47:57,776 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 02:47:57,777 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:57,777 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:47:57,777 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:57,777 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 02:47:57,777 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 02:47:57,777 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 02:47:57,777 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStates is initialized 02:47:57,777 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 02:47:57,777 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from null to Follower 02:47:57,777 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#204019730 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStates 02:47:57,779 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/shard-member-2:snapshot-read#683028364] 02:47:57,779 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 02:47:57,781 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:47:57,781 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 02:47:57,781 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:47:57,781 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 02:47:57,781 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 02:47:57,781 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:57,781 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:47:57,781 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 02:47:57,781 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStates is initialized 02:47:57,782 PM [cluster-test-shard-dispatcher-36] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:57,782 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStates is no longer the leader 02:47:57,782 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 02:47:57,782 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 02:47:57,782 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from null to Follower 02:47:57,797 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:47:57,801 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:47:57,807 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:47:57,811 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:47:57,811 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 02:47:57,811 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 02:47:57,811 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 02:47:57,811 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 02:47:57,811 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 02:47:57,811 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 02:47:57,811 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:47:57,811 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:57,811 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:57,811 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 02:47:57,811 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 02:47:57,811 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 02:47:57,811 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:57,818 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7916298763084727533] 02:47:57,819 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 02:47:57,823 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:47:57,823 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 02:47:57,824 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:47:57,824 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:47:57,838 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:57,849 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:57,854 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 02:47:57,868 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1312734710] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,870 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 02:47:57,882 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 02:47:57,882 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStates 02:47:57,883 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStates 02:47:57,885 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 02:47:57,892 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5cd63a69 02:47:57,892 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 02:47:57,893 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 02:47:57,893 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:57,894 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:47:57,894 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStates 02:47:57,897 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:47:57,899 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 02:47:57,901 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:47:57,902 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStates 02:47:57,902 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:47:57,902 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 02:47:57,902 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 02:47:57,902 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 02:47:57,902 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 02:47:57,903 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStates, persistent : true 02:47:57,903 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 02:47:57,904 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStates, persistent : true 02:47:57,904 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStates with journal batch size 1 02:47:57,905 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 02:47:57,908 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#1568409796] 02:47:57,908 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#1821642600 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStates 02:47:57,908 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#1675785772] 02:47:57,908 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1361299301] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,908 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1749279980] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,908 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#710708335] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,908 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#1821642600] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,908 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#1568409796] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,908 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#1675785772] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:47:57,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 02:47:57,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:47:57,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 02:47:57,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 02:47:57,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:57,911 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:47:57,911 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 02:47:57,911 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStates is initialized 02:47:57,912 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:57,912 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStates is no longer the leader 02:47:57,912 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 02:47:57,912 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 02:47:57,912 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 02:47:57,912 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from null to Follower 02:47:57,914 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 02:47:57,914 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:47:57,914 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 02:47:57,914 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:47:57,914 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 02:47:57,914 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 02:47:57,914 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:57,914 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:47:57,915 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:57,915 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStates is no longer the leader 02:47:57,915 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 02:47:57,915 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 02:47:57,915 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStates is initialized 02:47:57,916 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:57,917 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#-618490443 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStates 02:47:57,917 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 02:47:57,917 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 02:47:57,918 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from null to Follower 02:47:57,918 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 02:47:57,918 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStates 02:47:57,919 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#-618490443] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,919 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 02:47:57,919 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:47:57,920 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:57,920 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 02:47:57,920 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@9698af7 02:47:57,920 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:47:57,920 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 02:47:57,920 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:57,921 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStates, persistent : true 02:47:57,922 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#-1495047543 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:57,922 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/shard-member-3:snapshot-read#-1935084910] 02:47:57,923 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 02:47:57,923 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:57,923 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:57,925 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1962556166] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,925 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#-1495047543] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:47:57,925 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-3-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1962556166] 02:47:57,926 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStates 02:47:57,926 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:47:57,926 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 02:47:57,926 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:47:57,927 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 02:47:57,927 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 02:47:57,927 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:57,927 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:47:57,927 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:57,927 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 02:47:57,927 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 02:47:57,927 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:57,927 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 02:47:57,927 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 02:47:57,927 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-3-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#320143463] 02:47:57,928 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 02:47:57,928 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStates is initialized 02:47:57,928 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 02:47:57,928 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from null to Follower 02:47:57,928 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStates, persistent : true 02:47:57,929 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#1287782609 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStates 02:47:57,929 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/shard-member-3:snapshot-read#1821990519] 02:47:57,929 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 02:47:57,930 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:47:57,930 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 02:47:57,930 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:47:57,931 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 02:47:57,931 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 02:47:57,931 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:47:57,931 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:47:57,931 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 02:47:57,931 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStates is initialized 02:47:57,931 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:57,931 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStates is no longer the leader 02:47:57,931 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 02:47:57,931 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 02:47:57,931 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 02:47:57,931 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from null to Follower 02:47:58,526 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 02:47:58,527 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:47:58,527 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:47:58,528 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:47:58,528 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:47:58,528 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 02:47:58,528 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:47:58,529 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 02:47:58,529 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 02:47:58,529 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 02:47:58,529 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 02:47:58,529 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 02:47:58,529 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 02:47:58,543 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:47:58,543 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 02:47:58,543 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 02:47:58,543 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 02:47:58,543 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 02:47:58,543 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:47:58,543 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:47:58,543 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 02:47:58,543 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:58,543 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 02:47:58,543 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:58,544 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 02:47:58,544 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 02:47:58,544 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 02:47:58,544 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 02:47:58,544 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:58,544 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 02:47:58,544 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 02:47:58,544 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:47:58,544 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:58,544 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:58,544 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 02:47:58,544 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 02:47:58,544 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 02:47:58,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:47:58,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 02:47:58,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStates] 02:47:58,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 02:47:58,674 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 02:47:58,674 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:47:58,674 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 02:47:58,674 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStates] 02:47:58,674 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 02:47:58,674 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 02:47:58,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-oper_testFlipMemberVotingStates 02:47:58,675 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-config_testFlipMemberVotingStates 02:47:58,677 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 02:47:58,677 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 02:47:58,677 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:58,677 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 02:47:58,677 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:58,677 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 02:47:58,677 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:58,677 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:58,677 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 02:47:58,677 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 02:47:58,683 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:47:58,684 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 02:47:58,684 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStates] 02:47:58,684 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 02:47:58,684 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 02:47:58,684 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-config_testFlipMemberVotingStates 02:47:58,684 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 02:47:58,684 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:58,684 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 02:47:58,684 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:58,684 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 02:47:58,691 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:47:58,691 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:47:58,691 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Cluster state: [] 02:47:58,691 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:47:58,691 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:47:58,691 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 02:47:58,692 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 02:47:58,691 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:47:58,692 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Cluster state: [] 02:47:58,692 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:47:58,692 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:47:58,692 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 02:47:58,692 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 02:47:58,692 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 02:47:58,691 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 02:47:58,692 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 02:47:58,692 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:47:58,693 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:47:58,693 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 02:47:58,693 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 02:47:58,693 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 02:47:58,693 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 02:47:58,692 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 02:47:58,693 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 02:47:58,703 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 02:47:58,703 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 02:47:58,703 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 02:47:58,703 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 02:47:58,703 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 02:47:58,703 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 02:47:58,704 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 02:47:58,704 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 02:47:58,704 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:58,704 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:58,704 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:58,704 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:58,704 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:47:58,704 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 02:47:58,704 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:58,704 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 02:47:58,704 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 02:47:58,704 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:58,704 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 02:47:58,704 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-3-shard-people-oper_testFlipMemberVotingStates] 02:47:58,704 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:47:58,704 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:58,704 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 02:47:58,704 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 02:47:58,704 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 02:47:58,704 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 02:47:58,704 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 02:47:58,705 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:47:58,705 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:58,705 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:58,705 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 02:47:58,705 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 02:47:58,705 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 02:47:58,705 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:58,705 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 02:47:58,705 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:58,704 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:58,706 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:58,706 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,706 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,706 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 02:47:58,706 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:58,706 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 02:47:58,706 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:58,706 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,706 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,706 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,706 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,706 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 02:47:58,706 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,706 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,706 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 02:47:58,706 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:58,706 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:58,706 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,706 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,707 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 02:47:58,707 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,707 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,707 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 02:47:58,707 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 02:47:58,707 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 02:47:58,708 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,708 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to PreLeader 02:47:58,708 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,708 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to PreLeader 02:47:58,708 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,708 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to PreLeader 02:47:58,708 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,708 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,709 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,710 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 02:47:58,710 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 02:47:58,711 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,712 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 02:47:58,712 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 02:47:58,712 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 02:47:58,713 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 02:47:58,713 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 02:47:58,713 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 02:47:58,713 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 02:47:58,713 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 02:47:58,713 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 02:47:58,713 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,714 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done false 02:47:58,714 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done true 02:47:58,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:47:58,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 02:47:58,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStates] 02:47:58,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:58,714 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 02:47:58,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:58,714 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 02:47:58,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:58,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 02:47:58,714 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:58,714 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 02:47:58,715 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 02:47:58,716 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 02:47:58,716 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,716 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 02:47:58,716 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 02:47:58,717 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 02:47:58,717 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,717 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:47:58,717 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 02:47:58,717 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 02:47:58,717 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 02:47:58,717 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 02:47:58,717 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,717 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 02:47:58,717 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,717 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 02:47:58,717 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done false 02:47:58,717 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done true 02:47:58,719 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 02:47:58,720 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 02:47:58,720 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,720 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 02:47:58,720 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,720 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 02:47:58,720 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 02:47:58,720 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,721 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 02:47:58,721 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,721 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 02:47:58,721 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,721 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,721 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 02:47:58,721 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,721 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,721 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 02:47:58,721 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,721 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 02:47:58,721 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,721 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 02:47:58,721 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,721 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,721 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 02:47:58,721 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,721 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 02:47:58,721 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,722 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,722 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,722 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 02:47:58,722 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,722 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,722 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 02:47:58,723 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 02:47:58,724 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 02:47:58,724 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:47:58,724 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 02:47:58,724 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:47:58,724 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:47:58,724 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 02:47:58,724 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 02:47:58,724 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:58,724 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 02:47:58,725 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,725 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,725 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:47:58,725 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,725 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,725 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:58,725 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,726 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,726 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,726 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:47:58,726 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,726 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,726 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:58,726 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,726 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,726 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done false 02:47:58,727 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,727 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,727 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:47:58,727 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,727 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,727 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:58,727 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,727 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,727 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,727 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 02:47:58,728 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done true 02:47:58,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 02:47:58,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,728 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,728 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,728 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 02:47:58,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,728 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 02:47:58,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 02:47:58,729 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,729 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 02:47:58,729 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 02:47:58,729 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 02:47:58,729 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,729 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 02:47:58,729 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 02:47:58,729 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,729 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 02:47:58,729 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,729 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 02:47:58,729 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,729 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:58,730 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:47:58,730 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 02:47:58,730 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:58,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,730 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:58,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:47:58,730 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 02:47:58,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,730 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,730 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,730 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:58,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,730 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to PreLeader 02:47:58,730 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 02:47:58,730 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,730 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,730 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,731 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,732 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,732 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,732 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:47:58,732 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,732 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,732 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,732 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,732 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:47:58,732 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,732 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,732 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:58,732 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,732 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:58,732 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,732 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,732 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,732 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 02:47:58,732 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 02:47:58,733 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:58,733 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,733 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 02:47:58,733 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done false 02:47:58,733 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done true 02:47:58,735 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 02:47:58,735 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,735 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:58,735 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,735 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:58,736 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 02:47:58,736 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,736 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 02:47:58,736 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 02:47:58,736 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 02:47:58,736 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,736 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 02:47:58,736 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 02:47:58,736 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,736 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 02:47:58,736 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:47:58,736 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:58,736 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 02:47:58,736 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]] - lastApplied: 1, lastIndex: 1 02:47:58,737 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,737 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 02:47:58,737 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 02:47:58,737 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:47:58,737 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:47:58,737 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 02:47:58,737 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:58,737 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 02:47:58,737 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 02:47:58,737 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from PreLeader to Leader 02:47:58,737 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 02:47:58,737 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,737 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 02:47:58,737 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 02:47:58,737 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 02:47:58,737 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,738 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 02:47:58,738 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 02:47:58,738 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,738 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 02:47:58,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 02:47:58,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,738 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:47:58,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 02:47:58,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 02:47:58,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 02:47:58,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,738 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]] - lastApplied: 1, lastIndex: 1 02:47:58,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 02:47:58,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 02:47:58,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 02:47:58,738 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 02:47:58,738 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:47:58,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:47:58,738 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:47:58,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]] - lastApplied: 1, lastIndex: 1 02:47:58,738 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 02:47:58,738 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:58,738 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 02:47:58,738 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 02:47:58,738 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from PreLeader to Leader 02:47:58,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testFlipMemberVotingStates, member-2-shard-people-oper_testFlipMemberVotingStates] 02:47:58,738 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 02:47:58,738 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:47:58,738 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 02:47:58,739 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 02:47:58,739 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:47:58,739 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 02:47:58,739 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:58,739 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 02:47:58,739 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from PreLeader to Leader 02:47:58,739 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 02:47:58,740 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 02:47:58,740 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,741 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,741 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:58,741 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 1 02:47:58,741 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:47:58,741 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 02:47:58,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 02:47:58,741 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:47:58,741 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,741 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 02:47:58,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 02:47:58,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,742 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:58,742 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 02:47:58,742 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,742 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,743 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 02:47:58,744 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 1 02:47:58,744 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:47:58,744 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 02:47:58,744 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 02:47:58,744 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,744 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 02:47:58,744 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 02:47:58,744 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,744 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 02:47:58,745 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 02:47:58,745 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 02:47:58,745 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,745 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:47:58,746 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 1 02:47:58,746 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:47:58,746 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 02:47:58,746 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 02:47:58,746 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,746 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 02:47:58,746 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 02:47:58,746 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,746 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 02:47:58,747 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 02:47:58,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 1 02:47:58,747 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 02:47:58,747 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:47:58,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:47:58,747 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 02:47:58,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 02:47:58,747 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 02:47:58,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 02:47:58,747 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,747 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 02:47:58,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 02:47:58,747 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 02:47:58,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 02:47:58,747 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,747 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 02:47:58,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 02:47:58,753 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 02:47:58,754 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:47:58,754 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 02:47:58,754 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 02:47:58,754 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,754 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 02:47:58,754 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 02:47:58,754 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,754 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 02:47:58,754 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 02:47:58,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:58,758 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:47:58,758 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,758 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:47:58,758 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,758 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,758 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:58,758 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,759 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:47:58,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 02:47:58,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:47:58,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 02:47:58,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 02:47:58,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 02:47:58,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 02:47:58,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 02:47:58,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 02:47:58,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:47:58,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]] - lastApplied: 1, lastIndex: 1 02:47:58,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 02:47:58,762 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:47:58,762 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:47:58,762 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 02:47:58,762 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:58,762 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 02:47:58,762 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 02:47:58,762 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from PreLeader to Leader 02:47:58,762 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 02:47:58,762 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 02:47:58,762 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 02:47:58,762 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 02:47:58,762 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 02:47:58,762 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 02:47:58,763 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#937717968] 02:47:58,763 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:47:58,763 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:58,764 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:58,764 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 02:47:58,764 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:47:58,764 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 02:47:58,764 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 02:47:58,764 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,764 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 02:47:58,764 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 02:47:58,764 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,764 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 02:47:58,764 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 02:47:58,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 02:47:58,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:47:58,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 02:47:58,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 02:47:58,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 02:47:58,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 02:47:58,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:47:58,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 02:47:58,774 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 02:47:58,774 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 02:47:58,774 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 02:47:58,774 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 02:47:58,774 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 02:47:58,776 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 02:47:58,776 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 02:47:58,776 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:47:58,776 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 02:47:58,776 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 02:47:58,776 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 02:47:58,778 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 02:47:58,778 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:58,779 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 02:47:58,779 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 02:47:58,779 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates)] 02:47:58,779 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 02:47:58,779 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 02:47:58,779 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 02:47:58,779 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 02:47:58,779 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:58,779 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates)] 02:47:58,779 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates)] 02:47:58,780 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 02:47:58,780 PM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 02:47:58,780 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 02:47:58,780 PM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates)] 02:47:58,791 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testFlipMemberVotingStates=false, member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 02:47:58,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 02:47:58,793 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:47:58,793 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:47:58,794 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 02:47:58,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 02:47:58,794 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]] 02:47:58,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]] 02:47:58,794 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,794 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,794 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 02:47:58,794 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=50136ca7-5f63-4bcd-b0b7-9f21bb9ee020}, logIndex: 2, isSendImmediate: true 02:47:58,794 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 02:47:58,794 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:47:58,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=3e296c70-0058-4c38-a06c-d888b9f8164d}, logIndex: 2, isSendImmediate: true 02:47:58,794 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:47:58,794 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 02:47:58,794 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:47:58,794 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 02:47:58,794 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]] 02:47:58,794 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 02:47:58,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:47:58,794 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 02:47:58,794 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,794 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,794 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]] 02:47:58,794 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,794 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=932ebd1d-df90-4ad3-9acb-ed2e6a9e456a}, logIndex: 2, isSendImmediate: true 02:47:58,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,795 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:47:58,795 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,795 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 02:47:58,795 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:47:58,795 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 02:47:58,795 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,795 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,795 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:47:58,795 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=bb10b695-28b8-4ca2-8bb9-dd5fe59b7b93}, logIndex: 2, isSendImmediate: true 02:47:58,795 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 02:47:58,795 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:47:58,795 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testFlipMemberVotingStates=false, member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] 02:47:58,795 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 02:47:58,795 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,795 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,795 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:47:58,795 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:58,795 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] 02:47:58,795 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] 02:47:58,795 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,795 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:47:58,795 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:58,795 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,795 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] 02:47:58,796 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,796 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,801 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,801 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,801 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,801 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,801 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:47:58,801 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:47:58,801 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,801 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,801 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,801 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,802 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 02:47:58,802 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:58,802 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,802 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 02:47:58,802 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:58,802 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,802 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,803 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,803 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,803 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,803 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,803 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,803 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,803 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,803 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:47:58,804 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:47:58,804 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,804 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,804 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,804 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:58,804 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,804 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 02:47:58,804 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:47:58,804 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 02:47:58,804 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:58,804 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:58,804 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,804 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:58,804 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,804 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:47:58,805 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,805 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,805 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:58,806 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:47:58,806 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:58,806 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:47:58,808 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,808 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,808 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:47:58,808 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,808 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,808 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,808 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,808 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:47:58,808 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 02:47:58,808 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 3 02:47:58,808 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,808 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,808 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,809 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 02:47:58,809 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 02:47:58,809 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 3 02:47:58,809 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,809 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 02:47:58,809 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,809 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,810 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,810 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,810 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:47:58,810 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,810 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,810 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 02:47:58,810 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 3 02:47:58,810 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,810 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 02:47:58,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:58,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 02:47:58,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Setting last applied to 2 02:47:58,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 02:47:58,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 02:47:58,811 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:58,811 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 02:47:58,811 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Setting last applied to 2 02:47:58,811 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 02:47:58,811 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,811 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 02:47:58,811 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,811 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 02:47:58,811 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 02:47:58,811 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 02:47:58,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 02:47:58,811 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 02:47:58,811 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 02:47:58,811 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 02:47:58,811 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 02:47:58,812 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,812 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 02:47:58,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 02:47:58,812 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@429173b2 02:47:58,812 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@429173b2 02:47:58,812 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:47:58,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7e4b1fc1 02:47:58,812 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:47:58,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7e4b1fc1 02:47:58,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:47:58,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:47:58,812 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,813 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,813 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,813 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 02:47:58,813 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 02:47:58,813 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:58,813 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:58,813 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:47:58,813 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,813 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,814 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 02:47:58,814 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 3 02:47:58,814 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:58,814 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:58,814 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:58,814 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 02:47:58,814 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:58,814 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 02:47:58,814 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 02:47:58,814 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 02:47:58,814 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:58,816 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:58,816 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 02:47:58,816 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Setting last applied to 2 02:47:58,816 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 02:47:58,816 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,816 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 02:47:58,816 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,816 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 02:47:58,816 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 02:47:58,816 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:58,816 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 02:47:58,816 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 02:47:58,816 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Setting last applied to 2 02:47:58,816 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 02:47:58,816 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,816 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:58,816 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 02:47:58,816 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@289eab3e 02:47:58,816 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@289eab3e 02:47:58,816 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:47:58,816 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,816 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:47:58,817 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,817 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 02:47:58,817 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 02:47:58,817 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 02:47:58,817 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,817 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:58,817 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 02:47:58,817 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@a897922 02:47:58,817 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 02:47:58,817 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@a897922 02:47:58,817 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 02:47:58,817 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:47:58,817 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:47:58,817 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 02:47:58,817 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 02:47:58,817 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,817 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 02:47:58,817 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 02:47:58,817 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 02:47:58,817 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 02:47:58,817 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,817 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 02:47:58,818 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#937717968] 02:47:58,818 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:47:58,819 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:58,819 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 02:47:58,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 2 02:47:58,819 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 02:47:58,819 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 02:47:58,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 02:47:58,819 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 02:47:58,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 02:47:58,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 02:47:58,819 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,819 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 02:47:58,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 02:47:58,819 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 02:47:58,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 02:47:58,819 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates] 02:47:58,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates] 02:47:58,819 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:58,819 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:58,819 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:58,820 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#937717968] 02:47:58,820 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 02:47:58,820 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#2007631992] 02:47:58,820 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#2007631992] 02:47:58,820 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#338775429] 02:47:58,820 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 02:47:58,821 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1912470264] 02:47:58,821 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1912470264] 02:47:58,821 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#338775429] 02:47:58,821 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 02:47:58,821 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1254883418] 02:47:58,821 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1254883418] 02:47:58,821 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1332099890] 02:47:58,822 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:47:58,822 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1721748110] 02:47:58,822 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1721748110] 02:47:58,823 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1332099890] 02:47:58,823 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 02:47:58,823 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates] 02:47:58,823 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-682274936] 02:47:58,823 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:58,823 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-682274936] 02:47:58,824 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1192044054] 02:47:58,825 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 02:47:58,825 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#369041399] 02:47:58,825 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 2 02:47:58,825 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 02:47:58,825 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 02:47:58,825 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 02:47:58,825 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,825 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#369041399] 02:47:58,825 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 02:47:58,825 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 02:47:58,825 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates] 02:47:58,825 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:58,826 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1192044054] 02:47:58,826 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 02:47:58,827 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates] 02:47:58,827 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-config_testFlipMemberVotingStates] 02:47:58,827 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 02:47:58,828 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#1690572164] 02:47:58,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 02:47:58,828 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:58,828 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 2 02:47:58,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 02:47:58,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 02:47:58,828 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 02:47:58,828 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:58,828 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 02:47:58,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,828 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 02:47:58,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 02:47:58,828 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,828 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 02:47:58,828 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 02:47:58,828 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 02:47:58,828 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#1690572164] 02:47:58,828 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1312734710] 02:47:58,828 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 02:47:58,828 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 02:47:58,828 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 02:47:58,828 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 02:47:58,828 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,828 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:47:58,828 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 02:47:58,828 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 02:47:58,828 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1361299301] 02:47:58,829 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1361299301] 02:47:58,829 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1312734710] 02:47:58,829 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 02:47:58,829 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1749279980] 02:47:58,829 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1749279980] 02:47:58,829 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#710708335] 02:47:58,829 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 02:47:58,829 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1962556166] 02:47:58,829 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1962556166] 02:47:58,830 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#710708335] 02:47:58,830 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 02:47:58,830 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#320143463] 02:47:58,830 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#320143463] 02:47:58,831 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#937717968] 02:47:58,831 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:47:58,831 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:58,831 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:58,832 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 02:47:58,832 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 02:47:58,832 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 02:47:58,832 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 02:47:58,833 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 2 02:47:58,833 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 02:47:58,833 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 02:47:58,833 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 02:47:58,833 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,833 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 02:47:58,833 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 02:47:58,834 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-config_testFlipMemberVotingStates] 02:47:58,834 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 02:47:58,834 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:58,834 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 02:47:58,834 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 02:47:58,834 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 02:47:58,834 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:58,834 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 02:47:58,834 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 02:47:58,834 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates] 02:47:58,834 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:58,834 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-people-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 02:47:58,834 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 02:47:58,834 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,835 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,835 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 02:47:58,835 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 02:47:58,835 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 02:47:58,835 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 02:47:58,835 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,835 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,836 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Leader transfer complete 02:47:58,836 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 02:47:58,836 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-people-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 02:47:58,836 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 02:47:58,836 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,836 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 02:47:58,836 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 02:47:58,837 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,837 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,837 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,837 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Leader transfer complete 02:47:58,837 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 02:47:58,837 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 02:47:58,837 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 02:47:58,839 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 02:47:58,849 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 02:47:58,849 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 02:47:58,849 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-people-oper_testFlipMemberVotingStates 02:47:58,849 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 02:47:58,850 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 02:47:58,850 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:58,850 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStates is no longer the leader 02:47:58,850 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 02:47:58,850 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 02:47:58,850 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-cars-config_testFlipMemberVotingStates 02:47:58,850 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 02:47:58,850 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 02:47:58,850 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:58,850 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStates is no longer the leader 02:47:58,850 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:58,850 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:58,850 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 02:47:58,850 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:58,850 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:58,850 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 02:47:58,850 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 02:47:58,850 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 02:47:58,850 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-1-shard-people-oper_testFlipMemberVotingStates] 02:47:58,850 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStates, member-1-shard-cars-config_testFlipMemberVotingStates] 02:47:58,850 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,850 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,851 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,851 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,851 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 02:47:58,851 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:47:58,851 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:58,851 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 02:47:58,851 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:47:58,851 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-people-config_testFlipMemberVotingStates 02:47:58,851 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:58,851 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:58,851 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 02:47:58,851 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:58,851 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 02:47:58,851 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:58,851 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 02:47:58,851 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStates is no longer the leader 02:47:58,851 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 02:47:58,851 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:58,851 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,851 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,851 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 02:47:58,851 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 02:47:58,851 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:58,851 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 02:47:58,851 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 02:47:58,851 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-1-shard-people-config_testFlipMemberVotingStates, member-2-shard-people-config_testFlipMemberVotingStates] 02:47:58,851 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,851 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:58,852 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 02:47:58,852 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:58,852 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 02:47:58,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 02:47:58,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-1-shard-cars-oper_testFlipMemberVotingStates] 02:47:58,853 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,853 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,853 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:47:58,853 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:58,853 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:58,853 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 02:47:58,853 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,853 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 02:47:58,853 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 02:47:58,856 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 02:47:58,856 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 02:47:58,856 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 02:47:58,856 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 02:47:58,856 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 02:47:58,856 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 02:47:58,856 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 02:47:58,856 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 02:47:58,856 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 02:47:58,856 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 02:47:58,857 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 02:47:58,857 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:58,857 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,857 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,857 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,857 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:58,857 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 02:47:58,857 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 02:47:58,857 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 02:47:58,857 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 02:47:58,857 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 02:47:58,858 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 02:47:58,858 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 02:47:58,858 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 02:47:58,858 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 02:47:58,858 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:58,858 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 02:47:58,858 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 02:47:58,858 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 02:47:58,858 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 02:47:58,858 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:58,858 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 02:47:58,858 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 02:47:58,858 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:58,858 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 02:47:58,858 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 02:47:58,858 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:58,858 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 02:47:58,859 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 02:47:58,859 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 02:47:58,859 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 02:47:58,859 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 02:47:58,859 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 02:47:58,859 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:58,859 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 02:47:58,859 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 02:47:58,859 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 02:47:58,859 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:58,859 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 02:47:58,859 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:58,859 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Leader to Follower 02:47:58,859 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 02:47:58,859 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:58,859 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:58,859 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 02:47:58,860 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,860 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:58,860 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,860 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:58,860 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:58,860 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,860 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:58,862 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 02:47:58,862 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 02:47:58,863 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 02:47:58,863 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 02:47:58,881 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:58,932 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:58,982 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:59,33 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:59,83 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:59,134 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:59,178 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 02:47:59,178 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 02:47:59,178 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 02:47:59,179 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader changed to member-3-shard-people-oper_testFlipMemberVotingStates 02:47:59,179 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader changed to member-3-shard-cars-config_testFlipMemberVotingStates 02:47:59,179 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-oper_testFlipMemberVotingStates in 366.6 ms 02:47:59,179 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader changed to member-3-shard-people-config_testFlipMemberVotingStates 02:47:59,179 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 02:47:59,179 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,179 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-config_testFlipMemberVotingStates in 362.6 ms 02:47:59,179 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-config_testFlipMemberVotingStates in 366.7 ms 02:47:59,179 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 02:47:59,179 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 02:47:59,179 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:59,179 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 02:47:59,179 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,179 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:59,179 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-oper_testFlipMemberVotingStates in 362.5 ms 02:47:59,179 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 02:47:59,179 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 02:47:59,179 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 02:47:59,182 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,182 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,182 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 02:47:59,182 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 02:47:59,182 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 02:47:59,183 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:59,183 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:59,184 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:59,184 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:59,184 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:59,184 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 02:47:59,184 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:59,184 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:47:59,184 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:59,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#338775429] 02:47:59,185 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 02:47:59,185 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1912470264] 02:47:59,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1912470264] 02:47:59,186 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 02:47:59,186 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 02:47:59,186 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 02:47:59,186 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 02:47:59,187 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 02:47:59,187 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 02:47:59,187 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 02:47:59,187 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:47:59,187 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 02:47:59,187 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 02:47:59,187 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 02:47:59,187 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 02:47:59,187 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 02:47:59,187 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 02:47:59,187 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 02:47:59,188 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:59,188 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 02:47:59,189 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:47:59,189 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 02:47:59,189 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 02:47:59,189 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]] 02:47:59,189 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 02:47:59,189 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,189 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,189 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=bcd40f6c-a468-4503-bcf3-beb05f76874c}, logIndex: 3, isSendImmediate: true 02:47:59,189 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 02:47:59,189 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 02:47:59,189 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:59,189 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,189 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 02:47:59,189 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 02:47:59,189 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:47:59,189 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:59,189 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 02:47:59,189 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,190 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]] 02:47:59,190 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,190 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] 02:47:59,190 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,190 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-oper_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:59,190 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 02:47:59,190 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 02:47:59,190 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=728c6a52-0ec0-4564-8500-f320d1a94fa6}, logIndex: 3, isSendImmediate: true 02:47:59,190 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:47:59,190 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 02:47:59,190 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 02:47:59,190 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 02:47:59,190 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 02:47:59,190 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]] 02:47:59,190 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 02:47:59,190 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,190 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,190 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,190 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=0987d943-8f97-4442-8aa9-19f7dd33b2b8}, logIndex: 3, isSendImmediate: true 02:47:59,190 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 02:47:59,190 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-config_testFlipMemberVotingStates 02:47:59,190 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 02:47:59,190 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-config_testFlipMemberVotingStates 02:47:59,190 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,190 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,190 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] 02:47:59,190 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 02:47:59,190 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 02:47:59,191 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,191 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 02:47:59,191 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] 02:47:59,191 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:47:59,191 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 02:47:59,191 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]] 02:47:59,191 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,191 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,191 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=f74a42b4-ade5-4136-ab1b-83a1cd62c71f}, logIndex: 3, isSendImmediate: true 02:47:59,191 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 02:47:59,191 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 02:47:59,191 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,191 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-config_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 02:47:59,191 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,191 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 02:47:59,191 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 02:47:59,191 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 02:47:59,191 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 02:47:59,191 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-oper_testFlipMemberVotingStates 02:47:59,191 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,191 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-config_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 02:47:59,191 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 02:47:59,191 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,191 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,191 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 02:47:59,191 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] 02:47:59,191 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 02:47:59,191 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-oper_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 02:47:59,191 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 02:47:59,192 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 02:47:59,192 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 02:47:59,192 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#937717968] 02:47:59,193 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,193 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-oper_testFlipMemberVotingStates 02:47:59,193 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-config_testFlipMemberVotingStates 02:47:59,193 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:59,193 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:47:59,193 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-config_testFlipMemberVotingStates 02:47:59,194 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:59,194 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,195 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,195 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:59,195 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 02:47:59,195 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:59,195 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:59,195 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:59,195 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:47:59,195 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,196 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,196 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,196 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 02:47:59,196 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:59,196 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 02:47:59,196 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:59,196 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:59,196 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:59,197 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,197 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,197 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,197 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,198 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,198 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 02:47:59,198 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Setting last applied to 3 02:47:59,198 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 02:47:59,198 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,198 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 02:47:59,198 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,199 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 02:47:59,199 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 02:47:59,199 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Setting last applied to 3 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,199 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@cb76255 02:47:59,199 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@cb76255 02:47:59,199 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:47:59,199 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [INFO] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 02:47:59,199 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 02:47:59,199 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 02:47:59,199 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,199 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@55fe93ef 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-3-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@55fe93ef 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:47:59,199 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 02:47:59,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Setting last applied to 3 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,200 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 02:47:59,200 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,200 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,200 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 02:47:59,200 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@27d65cd0 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@27d65cd0 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,200 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,200 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:59,200 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:47:59,200 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,200 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 02:47:59,200 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:47:59,201 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,202 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,202 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 02:47:59,202 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:59,202 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:59,202 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,202 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:59,202 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:47:59,202 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,202 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,202 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 02:47:59,202 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:59,202 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:59,202 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,202 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,202 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Setting last applied to 3 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [INFO] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 02:47:59,203 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,203 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4973d931 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-3-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4973d931 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 02:47:59,204 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,205 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 02:47:59,205 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 02:47:59,205 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 02:47:59,205 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 02:47:59,205 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,205 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 02:47:59,205 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 02:47:59,206 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 3 02:47:59,206 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 02:47:59,206 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 02:47:59,206 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 02:47:59,206 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,206 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 02:47:59,206 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 02:47:59,206 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates] 02:47:59,206 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates] 02:47:59,206 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,206 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,206 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 02:47:59,207 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 02:47:59,207 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 02:47:59,207 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 02:47:59,207 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,207 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 02:47:59,207 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 02:47:59,207 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates] 02:47:59,207 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,208 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,208 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,208 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 02:47:59,208 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 02:47:59,208 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,209 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,209 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 02:47:59,209 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 02:47:59,209 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,209 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,209 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 02:47:59,209 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 02:47:59,209 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,209 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,209 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Leader transfer complete 02:47:59,209 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 02:47:59,209 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,209 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 02:47:59,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 02:47:59,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 02:47:59,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 02:47:59,210 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,210 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:59,210 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,210 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 02:47:59,210 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:59,210 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 02:47:59,210 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:59,210 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 02:47:59,210 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 02:47:59,210 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 02:47:59,210 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:59,210 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:59,210 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 3 02:47:59,210 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 02:47:59,210 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:59,210 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 02:47:59,210 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 02:47:59,210 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 02:47:59,210 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,210 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 02:47:59,210 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates] 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates] 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 3 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates] 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,211 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,211 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,211 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-config_testFlipMemberVotingStates] 02:47:59,212 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,212 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 02:47:59,212 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,212 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 02:47:59,212 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,212 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,212 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Leader transfer complete 02:47:59,212 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 02:47:59,214 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 3 02:47:59,214 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 02:47:59,214 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 02:47:59,214 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 02:47:59,214 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 02:47:59,214 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 02:47:59,214 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 02:47:59,214 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-config_testFlipMemberVotingStates] 02:47:59,214 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,221 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 02:47:59,221 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-oper_testFlipMemberVotingStates] 02:47:59,221 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:59,221 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 02:47:59,221 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 02:47:59,222 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-config_testFlipMemberVotingStates] 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-cars-config_testFlipMemberVotingStates 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-1-shard-cars-config_testFlipMemberVotingStates 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStates is no longer the leader 02:47:59,222 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-oper_testFlipMemberVotingStates] 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-people-oper_testFlipMemberVotingStates 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 02:47:59,222 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-1-shard-people-oper_testFlipMemberVotingStates 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStates is no longer the leader 02:47:59,223 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-config_testFlipMemberVotingStates] 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-people-config_testFlipMemberVotingStates 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-1-shard-people-config_testFlipMemberVotingStates 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStates is no longer the leader 02:47:59,223 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 02:47:59,223 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,223 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 02:47:59,223 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 02:47:59,223 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,223 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 02:47:59,225 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:47:59,225 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 02:47:59,226 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:47:59,226 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Cluster state: [] 02:47:59,226 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 02:47:59,226 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 02:47:59,226 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 02:47:59,226 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 02:47:59,226 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-cars-config_testFlipMemberVotingStates 02:47:59,226 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:47:59,226 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:47:59,226 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 02:47:59,226 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 02:47:59,226 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 02:47:59,226 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 02:47:59,226 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:47:59,226 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 02:47:59,226 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Cluster state: [] 02:47:59,226 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-people-oper_testFlipMemberVotingStates 02:47:59,226 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 02:47:59,226 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 02:47:59,226 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:47:59,226 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 02:47:59,226 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 02:47:59,226 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-people-config_testFlipMemberVotingStates 02:47:59,226 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:47:59,225 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 02:47:59,227 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 02:47:59,227 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 02:47:59,227 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 02:47:59,227 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:59,227 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:47:59,229 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 02:47:59,229 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 02:47:59,229 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-1-shard-cars-config_testFlipMemberVotingStates] 02:47:59,229 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,229 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 02:47:59,229 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,229 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 02:47:59,229 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 02:47:59,229 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:59,230 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testFlipMemberVotingStates, member-1-shard-people-oper_testFlipMemberVotingStates] 02:47:59,230 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:59,230 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 02:47:59,230 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 02:47:59,230 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,230 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 02:47:59,230 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-1-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 02:47:59,230 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,230 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 02:47:59,230 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,230 PM [cluster-test-shard-dispatcher-36] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:59,230 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:59,230 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 02:47:59,230 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,230 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,230 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,230 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 02:47:59,230 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 02:47:59,230 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:59,230 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 02:47:59,230 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:59,230 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 02:47:59,230 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-2-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,230 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 02:47:59,231 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 02:47:59,230 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 02:47:59,231 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 02:47:59,232 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-1-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 02:47:59,232 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,232 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,232 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 02:47:59,232 PM [cluster-test-shard-dispatcher-36] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:59,232 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:59,232 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 02:47:59,232 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,232 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 02:47:59,232 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 02:47:59,234 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,234 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-2-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 02:47:59,234 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-2-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 02:47:59,234 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,234 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 02:47:59,234 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 02:47:59,235 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-2-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 02:47:59,235 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 02:47:59,235 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-2-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,235 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-2-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:59,235 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 02:47:59,235 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,235 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 02:47:59,235 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 02:47:59,236 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries [leaderId=member-2-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 3 - switching to Follower 02:47:59,236 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 02:47:59,236 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 02:47:59,236 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,236 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStates is no longer the leader 02:47:59,236 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 02:47:59,236 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,236 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 02:47:59,237 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,237 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,237 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,237 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,237 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,237 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,237 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,237 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,237 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries [leaderId=member-2-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 3 - switching to Follower 02:47:59,238 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 02:47:59,238 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 02:47:59,238 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries [leaderId=member-2-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 3 - switching to Follower 02:47:59,238 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,238 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 02:47:59,238 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStates is no longer the leader 02:47:59,238 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 02:47:59,238 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 02:47:59,238 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,238 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,238 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStates is no longer the leader 02:47:59,238 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 02:47:59,238 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 02:47:59,238 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,238 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Leader to Follower 02:47:59,239 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 02:47:59,240 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 02:47:59,240 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 02:47:59,240 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 02:47:59,240 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries [leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 3 - switching to Follower 02:47:59,240 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 02:47:59,241 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 02:47:59,241 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,241 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 02:47:59,241 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 02:47:59,241 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,241 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 02:47:59,245 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:59,246 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#937717968] 02:47:59,246 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 02:47:59,246 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#2007631992] 02:47:59,246 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#2007631992] 02:47:59,246 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#338775429] 02:47:59,246 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 02:47:59,246 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1912470264] 02:47:59,246 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1912470264] 02:47:59,247 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#338775429] 02:47:59,247 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 02:47:59,247 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1254883418] 02:47:59,247 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1254883418] 02:47:59,247 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1332099890] 02:47:59,247 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:47:59,247 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1721748110] 02:47:59,247 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1721748110] 02:47:59,254 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1332099890] 02:47:59,254 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 02:47:59,254 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-682274936] 02:47:59,254 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-682274936] 02:47:59,255 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1192044054] 02:47:59,255 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 02:47:59,255 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#369041399] 02:47:59,255 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#369041399] 02:47:59,255 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-1192044054] 02:47:59,256 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 02:47:59,256 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#1690572164] 02:47:59,256 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#1690572164] 02:47:59,256 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1312734710] 02:47:59,256 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:47:59,256 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1361299301] 02:47:59,256 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#1361299301] 02:47:59,256 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1312734710] 02:47:59,256 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 02:47:59,256 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1749279980] 02:47:59,257 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1749279980] 02:47:59,257 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#710708335] 02:47:59,257 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 02:47:59,257 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1962556166] 02:47:59,257 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1962556166] 02:47:59,257 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#710708335] 02:47:59,257 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 02:47:59,257 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#320143463] 02:47:59,257 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#320143463] 02:47:59,258 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#937717968] 02:47:59,258 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:47:59,258 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:59,258 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1033066013] 02:47:59,258 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 02:47:59,259 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStates 02:47:59,259 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStates 02:47:59,259 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 02:47:59,259 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onShutDown 02:47:59,260 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onShutDown 02:47:59,260 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStates 02:47:59,260 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 02:47:59,261 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStates 02:47:59,261 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 02:47:59,261 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 02:47:59,261 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 02:47:59,261 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 02:47:59,262 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:59,262 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStates 02:47:59,262 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 02:47:59,262 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onShutDown 02:47:59,262 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onShutDown 02:47:59,262 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStates 02:47:59,262 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStates 02:47:59,262 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 02:47:59,262 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 02:47:59,263 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 02:47:59,263 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 02:47:59,265 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:47:59,265 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:47:59,273 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:47:59,275 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:47:59,275 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:47:59,275 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:47:59,278 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:47:59,290 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 02:47:59,291 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStates 02:47:59,292 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStates 02:47:59,292 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 02:47:59,292 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onShutDown 02:47:59,292 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 02:47:59,292 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onShutDown 02:47:59,292 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 02:47:59,292 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a301b3b 02:47:59,292 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2d649566 02:47:59,292 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-2-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a301b3b 02:47:59,292 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-2-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2d649566 02:47:59,292 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:47:59,292 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:47:59,292 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:47:59,292 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:47:59,292 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,292 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,293 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,293 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,293 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,293 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,297 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-people-config_testFlipMemberVotingStates] 02:47:59,297 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,298 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-config_testFlipMemberVotingStates] 02:47:59,298 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,301 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 02:47:59,301 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 02:47:59,301 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,301 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,301 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Leader transfer complete 02:47:59,301 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 02:47:59,301 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-people-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 02:47:59,301 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 02:47:59,301 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,302 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,302 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Leader transfer complete 02:47:59,302 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 02:47:59,304 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 02:47:59,304 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStates] 02:47:59,304 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 02:47:59,304 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 5 02:47:59,305 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-cars-config_testFlipMemberVotingStates 02:47:59,305 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 02:47:59,305 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,305 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 02:47:59,305 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 02:47:59,305 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStates] 02:47:59,305 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 02:47:59,306 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 5 02:47:59,306 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:59,306 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,306 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-people-config_testFlipMemberVotingStates 02:47:59,306 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 02:47:59,306 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 02:47:59,306 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,306 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 02:47:59,306 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:59,306 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,306 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 02:47:59,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:47:59,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Cluster state: [] 02:47:59,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 02:47:59,309 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than leader's term 4 - switching to Follower 02:47:59,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=null 02:47:59,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Leadership transfer in progress - processing RequestVote 02:47:59,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 3, lastTerm: 3 02:47:59,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 02:47:59,310 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:47:59,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 02:47:59,310 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Cluster state: [] 02:47:59,310 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 02:47:59,310 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than leader's term 4 - switching to Follower 02:47:59,310 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=null 02:47:59,310 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 02:47:59,310 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Leadership transfer in progress - processing RequestVote 02:47:59,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,310 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): In requestVote: RequestVote [term=5, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 3, lastTerm: 3 02:47:59,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStates is no longer the leader 02:47:59,310 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 02:47:59,310 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 02:47:59,310 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 02:47:59,310 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,310 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 02:47:59,310 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 02:47:59,310 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,310 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStates is no longer the leader 02:47:59,311 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 02:47:59,311 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,311 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Leader to Follower 02:47:59,312 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 02:47:59,312 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 02:47:59,312 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 02:47:59,312 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 02:47:59,312 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 02:47:59,312 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 02:47:59,312 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,312 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,312 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,312 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,312 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 02:47:59,313 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:59,313 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 02:47:59,313 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:59,313 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:59,313 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:59,313 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 02:47:59,313 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 02:47:59,313 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,313 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,313 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 02:47:59,313 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 02:47:59,313 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 02:47:59,315 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 02:47:59,315 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,315 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 02:47:59,315 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: leader changed to member-1-shard-people-config_testFlipMemberVotingStates 02:47:59,315 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,315 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: leader changed to member-1-shard-cars-config_testFlipMemberVotingStates 02:47:59,315 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 02:47:59,315 PM [cluster-test-shard-dispatcher-15] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-people-config_testFlipMemberVotingStates in 22.99 ms 02:47:59,315 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: leader transfer succeeded - sending PoisonPill 02:47:59,315 PM [cluster-test-shard-dispatcher-33] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-config_testFlipMemberVotingStates in 23.20 ms 02:47:59,315 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: leader transfer succeeded - sending PoisonPill 02:47:59,315 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 02:47:59,315 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 02:47:59,316 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStates 02:47:59,316 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStates 02:47:59,316 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 02:47:59,316 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 02:47:59,316 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 02:47:59,316 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 02:47:59,317 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,317 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,317 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 02:47:59,317 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,317 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,317 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:59,317 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStates 02:47:59,317 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 02:47:59,318 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onShutDown 02:47:59,318 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 02:47:59,318 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onShutDown 02:47:59,318 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 02:47:59,318 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5d582536 02:47:59,318 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,318 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5d582536 02:47:59,318 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:47:59,318 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:47:59,318 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@c1e0f6 02:47:59,318 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,318 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-2-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@c1e0f6 02:47:59,318 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:47:59,318 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,318 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:47:59,318 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,318 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,318 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,321 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates] 02:47:59,321 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,321 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-people-oper_testFlipMemberVotingStates] 02:47:59,321 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:47:59,323 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-people-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 02:47:59,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 02:47:59,323 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 02:47:59,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 02:47:59,323 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,323 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 02:47:59,323 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 02:47:59,323 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 02:47:59,323 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 02:47:59,326 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 02:47:59,326 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStates] 02:47:59,326 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:59,326 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 5 02:47:59,326 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 02:47:59,326 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStates] 02:47:59,326 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 02:47:59,326 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:59,326 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 5 02:47:59,326 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 02:47:59,326 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,326 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 02:47:59,326 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-people-oper_testFlipMemberVotingStates 02:47:59,326 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:59,326 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 02:47:59,326 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,327 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 02:47:59,327 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,327 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 02:47:59,327 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 02:47:59,327 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,327 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 02:47:59,328 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:47:59,328 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Cluster state: [] 02:47:59,328 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 02:47:59,328 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than leader's term 4 - switching to Follower 02:47:59,328 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=null 02:47:59,328 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:47:59,328 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Leadership transfer in progress - processing RequestVote 02:47:59,328 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 3, lastTerm: 3 02:47:59,328 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:59,328 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Cluster state: [] 02:47:59,328 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 02:47:59,328 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 02:47:59,328 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than leader's term 4 - switching to Follower 02:47:59,328 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=null 02:47:59,329 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Leadership transfer in progress - processing RequestVote 02:47:59,329 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): In requestVote: RequestVote [term=5, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 3, lastTerm: 3 02:47:59,329 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 02:47:59,329 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 02:47:59,329 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 02:47:59,329 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,329 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 02:47:59,329 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 02:47:59,329 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 02:47:59,329 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 02:47:59,329 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,329 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStates is no longer the leader 02:47:59,329 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 02:47:59,329 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 02:47:59,329 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 02:47:59,329 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 02:47:59,331 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 02:47:59,331 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 02:47:59,331 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 02:47:59,331 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 02:47:59,331 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 02:47:59,331 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-3-shard-people-oper_testFlipMemberVotingStates] 02:47:59,331 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,331 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,331 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,331 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,331 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 02:47:59,331 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:59,331 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 02:47:59,331 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:59,331 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 02:47:59,331 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 02:47:59,331 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 02:47:59,331 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,332 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 02:47:59,332 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 02:47:59,332 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,332 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 02:47:59,332 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 02:47:59,333 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:59,333 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-1-shard-cars-oper_testFlipMemberVotingStates 02:47:59,333 PM [cluster-test-shard-dispatcher-15] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-oper_testFlipMemberVotingStates in 15.77 ms 02:47:59,333 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: leader transfer succeeded - sending PoisonPill 02:47:59,333 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 02:47:59,334 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: leader changed to member-1-shard-people-oper_testFlipMemberVotingStates 02:47:59,334 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,334 PM [cluster-test-shard-dispatcher-13] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-people-oper_testFlipMemberVotingStates in 16.02 ms 02:47:59,334 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: leader transfer succeeded - sending PoisonPill 02:47:59,334 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 02:47:59,334 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStates 02:47:59,334 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 02:47:59,334 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 02:47:59,334 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 02:47:59,334 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 02:47:59,334 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStates 02:47:59,334 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 02:47:59,335 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 02:47:59,335 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 02:47:59,335 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,335 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,336 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 02:47:59,336 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:47:59,336 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:47:59,336 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:47:59,545 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 02:48:00,284 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [7916298763084727533] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:48:00,355 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:00,356 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [8702847156426538529] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:48:00,357 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:00,360 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:48:00.364] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:00,385 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 02:48:00,386 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStates 02:48:00,386 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStates 02:48:00,386 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 02:48:00,386 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onShutDown 02:48:00,386 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onShutDown 02:48:00,386 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 02:48:00,387 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 02:48:00,387 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@9c2a480 02:48:00,387 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@9c2a480 02:48:00,387 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:48:00,387 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:48:00,387 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:48:00,387 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:48:00,388 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6f3bae05 02:48:00,388 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6f3bae05 02:48:00,388 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:48:00,388 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:48:00,388 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:48:00,388 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:48:00,391 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:48:00,391 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:48:01,463 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer aborted 02:48:01,463 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer aborted 02:48:01,464 PM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.077 s 02:48:01,464 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In unpauseLeader 02:48:01,464 PM [cluster-test-shard-dispatcher-34] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.076 s 02:48:01,464 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In unpauseLeader 02:48:01,464 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 02:48:01,464 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 02:48:01,464 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Leadership transfer expired 02:48:01,464 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Leadership transfer expired 02:48:01,465 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStates 02:48:01,465 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStates 02:48:01,465 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 02:48:01,465 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 02:48:01,466 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:48:01,466 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:48:01,467 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 02:48:01,467 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStates 02:48:01,467 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStates 02:48:01,467 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 02:48:01,468 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onShutDown 02:48:01,468 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 02:48:01,468 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onShutDown 02:48:01,468 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 02:48:01,468 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:48:01,468 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 02:48:01,468 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5075a849 02:48:01,468 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5075a849 02:48:01,468 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5451daa6 02:48:01,468 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:48:01,468 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5451daa6 02:48:01,468 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:48:01,468 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 02:48:01,468 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 02:48:01,468 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:48:01,468 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:48:01,469 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:48:01,469 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:48:02,564 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer aborted 02:48:02,564 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer aborted 02:48:02,564 PM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.096 s 02:48:02,564 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In unpauseLeader 02:48:02,564 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.096 s 02:48:02,564 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In unpauseLeader 02:48:02,564 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 02:48:02,564 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 02:48:02,564 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Leadership transfer expired 02:48:02,564 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Leadership transfer expired 02:48:02,565 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStates 02:48:02,566 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStates 02:48:02,566 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 02:48:02,566 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 02:48:02,566 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:48:02,566 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:48:02,569 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:02,569 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:02,573 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:48:02.577] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:02,598 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:48:02,611 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [20710172547531104] 02:48:02,612 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:48:02,614 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:48:02,614 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:48:02,618 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:02,618 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:02,636 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:02,638 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:02,641 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 02:48:02,641 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:48:02,641 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:48:02,642 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:48:02,653 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#799965621] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:02,654 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:48:02,659 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:48:02,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:02,660 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testModuleShardLeaderMovement 02:48:02,660 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:02,660 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:02,660 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 02:48:02,665 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 02:48:02,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6ad5589a 02:48:02,666 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 02:48:02,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:02,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:02,668 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:02,668 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:02,668 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement on shard member-1-shard-cars-config_testModuleShardLeaderMovement whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038] 02:48:02,669 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testModuleShardLeaderMovement 02:48:02,669 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testModuleShardLeaderMovement, persistent : true 02:48:02,669 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 02:48:02,669 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 02:48:02,669 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement on shard member-1-shard-people-config_testModuleShardLeaderMovement whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#-1169702651] 02:48:02,670 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:02,670 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-people-config_testModuleShardLeaderMovement, persistent : true 02:48:02,670 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#204789260 created and ready for shard:member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:02,670 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#1436721578] 02:48:02,670 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#204789260] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:02,670 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#-1169702651] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:02,670 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#1436721578] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:02,670 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 02:48:02,671 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#1093252724] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:02,671 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#1221426836] 02:48:02,671 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#1093252724 created and ready for shard:member-1-shard-people-config_testModuleShardLeaderMovement 02:48:02,671 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#1221426836] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:02,671 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-people-config_testModuleShardLeaderMovement with journal batch size 1 02:48:02,673 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:02,673 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:02,673 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:02,674 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 02:48:02,674 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 02:48:02,674 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 02:48:02,674 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 02:48:02,674 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:02,674 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 02:48:02,674 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:02,674 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:02,674 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 02:48:02,674 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 02:48:02,681 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 02:48:02,681 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testModuleShardLeaderMovement is initialized 02:48:02,681 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:02,681 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:02,681 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 02:48:02,681 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:02,681 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testModuleShardLeaderMovement is initialized 02:48:02,681 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testModuleShardLeaderMovement is no longer the leader 02:48:02,681 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 02:48:02,681 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 02:48:02,681 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-people-config_testModuleShardLeaderMovement 02:48:02,681 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 02:48:02,681 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:02,681 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:02,681 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testModuleShardLeaderMovement is no longer the leader 02:48:02,682 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 02:48:02,683 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 02:48:02,683 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 02:48:02,683 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 02:48:02,683 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:48:02,683 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:48:02,683 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 02:48:02,683 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 02:48:02,681 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from null to Follower 02:48:02,683 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 02:48:02,683 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 02:48:02,683 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 02:48:02,683 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 02:48:02,683 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:02,684 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 02:48:02,684 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: became leader with frontend state for [] 02:48:02,684 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 02:48:02,684 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 02:48:02,684 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testModuleShardLeaderMovement, leaderId=member-1-shard-people-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:02,684 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testModuleShardLeaderMovement from Candidate to Leader 02:48:02,684 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:02,717 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#799965621] 02:48:02,718 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:02,718 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038] 02:48:02,719 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:02,724 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:02,730 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 02:48:02,733 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1500800107] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:02,735 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:48:02,737 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:48:02,737 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:02,737 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testModuleShardLeaderMovement 02:48:02,738 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:02,738 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:02,738 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 02:48:02,739 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 02:48:02,740 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@204c8795 02:48:02,741 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 02:48:02,743 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:02,743 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:02,743 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 02:48:02,744 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#556203244] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:02,744 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/shard-member-1:snapshot-read#1648839704] 02:48:02,744 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#-336480404 created and ready for shard:member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:02,744 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 02:48:02,745 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#-336480404] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:02,745 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:02,745 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:02,746 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:02,746 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:02,746 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:02,746 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 02:48:02,746 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 02:48:02,746 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 02:48:02,747 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 02:48:02,747 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:02,747 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 02:48:02,747 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:02,747 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:02,747 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 02:48:02,747 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 02:48:02,746 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement on shard member-1-shard-cars-oper_testModuleShardLeaderMovement whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#556203244] 02:48:02,747 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testModuleShardLeaderMovement 02:48:02,748 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 02:48:02,748 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 02:48:02,748 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:48:02,748 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 02:48:02,748 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 02:48:02,748 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 02:48:02,749 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 02:48:02,749 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 02:48:02,749 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement on shard member-1-shard-people-oper_testModuleShardLeaderMovement whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#1582465931] 02:48:02,749 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 02:48:02,749 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testModuleShardLeaderMovement is initialized 02:48:02,749 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 02:48:02,749 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-oper_testModuleShardLeaderMovement, persistent : true 02:48:02,750 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:02,750 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/shard-member-1:snapshot-read#1078818341] 02:48:02,750 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#-1549316390 created and ready for shard:member-1-shard-people-oper_testModuleShardLeaderMovement 02:48:02,750 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testModuleShardLeaderMovement with journal batch size 1 02:48:02,750 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Candidate to Leader 02:48:02,751 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:02,751 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:02,751 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:02,751 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testModuleShardLeaderMovement is no longer the leader 02:48:02,751 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from null to Follower 02:48:02,751 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 02:48:02,751 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testModuleShardLeaderMovement is initialized 02:48:02,751 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 02:48:02,752 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 02:48:02,752 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-people-oper_testModuleShardLeaderMovement 02:48:02,752 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 02:48:02,752 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:02,752 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:02,752 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testModuleShardLeaderMovement is no longer the leader 02:48:02,752 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 02:48:02,752 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 02:48:02,752 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 02:48:02,752 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:48:02,752 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 02:48:02,752 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 02:48:02,752 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 02:48:02,753 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 02:48:02,753 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-people-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:02,753 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testModuleShardLeaderMovement from Candidate to Leader 02:48:02,753 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 02:48:02,793 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1500800107] 02:48:02,793 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 02:48:02,793 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#556203244] 02:48:02,806 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:48:02,824 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [6647913976749233398] 02:48:02,825 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:48:02,829 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:02,829 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:48:02,830 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:02,830 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:02,850 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:02,852 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:02,856 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 02:48:02,882 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1062349602] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:02,883 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 02:48:02,885 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 02:48:02,888 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 02:48:02,889 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@ad7e60d 02:48:02,889 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 02:48:02,890 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:02,890 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:02,892 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:02,894 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 02:48:02,896 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1892640736] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:02,896 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 02:48:02,897 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 02:48:02,900 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 02:48:02,901 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2f8d7d 02:48:02,905 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:48:02,905 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 02:48:02,906 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:02,909 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:48:02,910 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:02,910 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 02:48:02,911 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:02,911 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:02,926 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:48:02,944 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4044410884997460478] 02:48:02,944 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 02:48:02,946 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:02,946 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 02:48:02,946 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:02,946 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:02,961 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:02,963 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:02,967 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 02:48:02,979 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#140986663] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:02,980 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 02:48:02,993 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 02:48:02,998 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 02:48:03,0 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 02:48:03,13 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 02:48:03,15 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:48:03,13 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2d7a0e70 02:48:03,17 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:03,17 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:03,18 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:03,18 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:03,20 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:03,24 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 02:48:03,25 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1018933901] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:03,26 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 02:48:03,27 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 02:48:03,28 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:03,28 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 02:48:03,30 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 02:48:03,32 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4be9b3bb 02:48:03,32 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 02:48:03,33 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:03,635 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 02:48:03,635 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:48:03,637 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:03,637 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:03,637 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:03,637 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement 02:48:03,637 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:03,637 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:03,637 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement 02:48:03,637 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 02:48:03,637 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:03,637 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:03,638 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:03,638 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:03,638 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement 02:48:03,638 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 02:48:03,638 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:03,639 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement 02:48:03,639 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 02:48:03,639 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 02:48:03,639 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:03,639 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement 02:48:03,639 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:03,639 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement 02:48:03,639 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 02:48:03,639 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 02:48:03,651 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:03,651 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 02:48:03,651 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:03,651 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 02:48:03,651 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:03,651 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:03,651 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:03,651 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:03,870 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:03,870 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:03,870 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:03,870 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:03,870 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:03,870 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 02:48:03,870 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:03,870 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 02:48:03,939 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 02:48:03,939 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 02:48:03,940 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:03,940 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement] 02:48:03,944 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 02:48:03,945 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038, primaryVersion=13] 02:48:03,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038 for shard member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:03,949 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-config_testModuleShardLeaderMovement, persistent : true 02:48:03,950 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-847760943] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:03,951 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/shard-member-2:snapshot-read#1935990153] 02:48:03,951 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#-1543403679 created and ready for shard:member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:03,951 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#-1543403679] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:03,951 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/shard-member-2:snapshot-read#1935990153] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:03,951 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 02:48:03,952 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:03,952 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true], state: Idle 02:48:03,952 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:03,952 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 02:48:03,952 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 02:48:03,952 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:48:03,952 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testModuleShardLeaderMovement is initialized 02:48:03,952 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:03,952 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 02:48:03,952 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:03,952 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 1 02:48:03,953 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:48:03,953 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:03,953 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 02:48:03,953 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 02:48:03,953 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from null to Follower 02:48:03,953 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 02:48:03,953 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@21fadc6b, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testModuleShardLeaderMovement, ServerConfigPayload=null] 02:48:03,953 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 02:48:03,953 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 02:48:03,953 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testModuleShardLeaderMovement: 229 02:48:03,953 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement), Chunk: 1/1 02:48:03,954 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:48:03,954 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 02:48:03,954 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 02:48:03,954 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 02:48:03,954 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 02:48:03,955 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 02:48:03,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 02:48:03,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 02:48:03,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 02:48:03,955 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:03,956 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-config_testModuleShardLeaderMovement to state 02:48:03,956 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:03,956 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 02:48:03,956 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:03,957 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@81a3baf, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 02:48:03,957 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 02:48:03,957 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 02:48:03,957 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:48:03,957 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 02:48:03,957 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to -1 02:48:03,957 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 02:48:03,957 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 02:48:03,957 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 02:48:03,957 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#153663962] with name member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 02:48:03,957 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#153663962] 02:48:03,957 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 430.2 μs 02:48:03,958 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 02:48:03,958 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 02:48:03,958 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 02:48:03,958 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#153663962] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:03,958 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 02:48:03,958 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 383.0 μs 02:48:03,958 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 02:48:03,958 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 02:48:03,960 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 02:48:03,960 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 02:48:03,960 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 02:48:03,960 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testModuleShardLeaderMovement] 02:48:03,960 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]] 02:48:03,960 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:03,960 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:03,960 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=1ce38e74-50ca-48ab-b1c4-017dd3065089}, logIndex: 0, isSendImmediate: true 02:48:03,961 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:03,961 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:03,961 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 02:48:03,961 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 02:48:03,962 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:03,963 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 02:48:03,963 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 02:48:03,964 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:03,964 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 02:48:03,964 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:03,964 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038 02:48:03,964 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 02:48:03,964 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING]] 02:48:03,964 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 1 02:48:03,964 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:03,964 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:03,965 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:03,965 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 02:48:03,965 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:03,965 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 02:48:03,965 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:03,965 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:03,966 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 02:48:03,966 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 02:48:03,966 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 02:48:03,966 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:03,966 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:03,967 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 02:48:03,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 02:48:03,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 02:48:03,967 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 0, nextIndex: 1 02:48:03,967 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:48:03,967 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1062349602] 02:48:03,967 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Setting last applied to 0 02:48:03,967 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 02:48:03,967 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 02:48:03,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:03,967 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:03,968 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:03,968 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 02:48:03,968 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-847760943] 02:48:03,968 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-847760943] 02:48:03,969 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 0 02:48:03,969 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:03,969 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 0 02:48:03,969 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 02:48:03,969 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1892640736] 02:48:03,969 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 02:48:03,970 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 02:48:03,970 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 02:48:03,970 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 02:48:03,970 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 02:48:03,970 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 02:48:03,971 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 02:48:03,971 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:03,971 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement] 02:48:03,973 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 02:48:03,973 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#556203244, primaryVersion=13] 02:48:03,975 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#556203244 for shard member-2-shard-cars-oper_testModuleShardLeaderMovement 02:48:03,976 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#1719576922] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:03,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true], state: Idle 02:48:03,977 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-2-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 02:48:03,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 02:48:03,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:48:03,977 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testModuleShardLeaderMovement 02:48:03,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 1 02:48:03,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:48:03,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-2-shard-cars-oper_testModuleShardLeaderMovement 02:48:03,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 02:48:03,977 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6e003c1f, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testModuleShardLeaderMovement, ServerConfigPayload=null] 02:48:03,978 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 02:48:03,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 02:48:03,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: 229 02:48:03,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testModuleShardLeaderMovement, member-2-shard-cars-oper_testModuleShardLeaderMovement), Chunk: 1/1 02:48:03,978 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:48:03,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 02:48:03,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 02:48:03,978 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#-949764854 created and ready for shard:member-2-shard-cars-oper_testModuleShardLeaderMovement 02:48:03,978 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#-949764854] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:03,979 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/shard-member-2:snapshot-read#-245494385] 02:48:03,979 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 02:48:03,979 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 02:48:03,979 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/shard-member-2:snapshot-read#-245494385] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:03,979 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 02:48:03,980 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:03,980 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:03,980 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 02:48:03,981 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testModuleShardLeaderMovement is initialized 02:48:03,981 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:03,981 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 02:48:03,981 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 02:48:03,981 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 02:48:03,981 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 02:48:03,981 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 02:48:03,981 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 02:48:03,981 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:03,982 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement to state 02:48:03,982 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-oper_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:03,982 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7bf350ae, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 02:48:03,982 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 02:48:03,983 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 02:48:03,983 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 02:48:03,983 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:48:03,983 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 02:48:03,983 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 02:48:03,983 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to -1 02:48:03,983 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 02:48:03,983 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 02:48:03,983 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 02:48:03,983 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 02:48:03,983 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 02:48:03,983 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#168108106] 02:48:03,984 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#168108106] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:03,983 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#168108106] with name member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 02:48:03,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 1.154 ms 02:48:03,984 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 02:48:03,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 02:48:03,992 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 02:48:03,993 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 02:48:03,993 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 180.4 μs 02:48:03,993 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 02:48:03,993 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 02:48:03,994 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 02:48:03,994 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 02:48:03,994 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 02:48:03,994 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement] 02:48:03,994 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]] 02:48:03,994 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:03,994 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:03,995 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=a204d3a1-3995-47bb-87d7-04aa862df369}, logIndex: 0, isSendImmediate: true 02:48:03,995 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:03,995 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testModuleShardLeaderMovement 02:48:03,995 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 02:48:03,995 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 02:48:03,996 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:03,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 02:48:03,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 02:48:03,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:03,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 02:48:03,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:03,997 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#556203244 02:48:03,997 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 02:48:03,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING]] 02:48:03,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 1 02:48:03,997 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:03,998 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-oper_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:03,998 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testModuleShardLeaderMovement 02:48:03,998 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 02:48:03,998 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:03,998 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 02:48:03,998 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 02:48:03,998 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:03,999 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:03,999 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 0, nextIndex: 1 02:48:03,998 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1892640736] 02:48:03,999 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:48:03,999 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Setting last applied to 0 02:48:03,999 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 02:48:03,999 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 02:48:03,999 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:03,999 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 02:48:03,999 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 02:48:03,999 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 02:48:03,999 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:03,999 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 02:48:03,999 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 02:48:03,999 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#1719576922] 02:48:03,999 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:03,999 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 02:48:03,999 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 02:48:04,0 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#1719576922] 02:48:04,0 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 02:48:04,0 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 02:48:04,1 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:04,1 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement] 02:48:04,1 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 0 02:48:04,2 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:04,2 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 0 02:48:04,2 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 02:48:04,2 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 02:48:04,2 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 02:48:04,2 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 02:48:04,4 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 02:48:04,4 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038, primaryVersion=13] 02:48:04,7 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038 for shard member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,8 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-cars-config_testModuleShardLeaderMovement, persistent : true 02:48:04,8 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1112659353] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:04,9 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true], state: Idle 02:48:04,9 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 02:48:04,9 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 02:48:04,9 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#-34957027 created and ready for shard:member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,9 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/shard-member-3:snapshot-read#177482201] 02:48:04,9 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 3 02:48:04,10 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#-34957027] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:04,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:48:04,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,10 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/shard-member-3:snapshot-read#177482201] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:04,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 02:48:04,10 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@61a0fbbf, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testModuleShardLeaderMovement, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,10 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 02:48:04,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 02:48:04,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-3-shard-cars-config_testModuleShardLeaderMovement: 229 02:48:04,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement), Chunk: 1/1 02:48:04,11 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 02:48:04,11 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 3 02:48:04,11 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 02:48:04,11 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:04,11 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 02:48:04,11 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:04,11 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 02:48:04,11 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 02:48:04,11 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testModuleShardLeaderMovement is initialized 02:48:04,11 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:04,11 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 02:48:04,11 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 02:48:04,11 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 02:48:04,11 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from null to Follower 02:48:04,12 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]" message is greater than follower's term 0 - updating term 02:48:04,12 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 02:48:04,12 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,12 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 02:48:04,12 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:04,13 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-config_testModuleShardLeaderMovement to state 02:48:04,13 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:04,13 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 02:48:04,13 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:04,13 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1f74643a, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,13 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 02:48:04,14 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:48:04,14 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 02:48:04,14 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 02:48:04,14 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 02:48:04,14 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 02:48:04,14 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 02:48:04,14 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 02:48:04,14 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 02:48:04,14 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#1093384716] 02:48:04,14 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#1093384716] with name member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 02:48:04,14 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 02:48:04,14 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#1093384716] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:04,14 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 77.26 μs 02:48:04,15 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 799.0 μs 02:48:04,15 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 02:48:04,15 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 02:48:04,15 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 02:48:04,15 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 02:48:04,15 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 02:48:04,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 02:48:04,16 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 02:48:04,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 02:48:04,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-config_testModuleShardLeaderMovement] 02:48:04,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]] 02:48:04,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=1cf29a7c-3d47-4306-bd2f-c65a83972eac}, logIndex: 1, isSendImmediate: true 02:48:04,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:04,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 02:48:04,17 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:04,17 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:04,17 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 02:48:04,17 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 02:48:04,18 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 02:48:04,18 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 02:48:04,18 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:04,18 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 02:48:04,18 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,18 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,18 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 02:48:04,18 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 1 02:48:04,18 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:04,18 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:04,18 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:04,18 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:04,19 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,19 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 02:48:04,19 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 02:48:04,19 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 02:48:04,19 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:04,19 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 02:48:04,19 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 02:48:04,20 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038 02:48:04,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 02:48:04,20 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 02:48:04,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Setting last applied to 1 02:48:04,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 02:48:04,20 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 02:48:04,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 02:48:04,20 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 2 02:48:04,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 02:48:04,20 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:04,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,20 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:04,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 02:48:04,21 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,21 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 02:48:04,21 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 1 02:48:04,21 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:48:04,21 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 1 02:48:04,21 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 02:48:04,21 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 02:48:04,21 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 02:48:04,22 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 02:48:04,22 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,22 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,22 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 02:48:04,22 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:04,23 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 02:48:04,23 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 02:48:04,23 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 1 02:48:04,23 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:48:04,23 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 1 02:48:04,23 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 02:48:04,23 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 02:48:04,23 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 02:48:04,23 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 02:48:04,24 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 02:48:04,24 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 02:48:04,25 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#140986663] 02:48:04,24 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:04,26 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:04,27 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1112659353] 02:48:04,27 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1112659353] 02:48:04,27 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1018933901] 02:48:04,28 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 02:48:04,28 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 02:48:04,28 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 02:48:04,28 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 02:48:04,28 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:04,28 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement] 02:48:04,28 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 02:48:04,29 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 02:48:04,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#556203244, primaryVersion=13] 02:48:04,32 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-3-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 02:48:04,33 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 02:48:04,33 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#-416294505] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:04,33 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/shard-member-3:snapshot-read#-1675202761] 02:48:04,33 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#122231255 created and ready for shard:member-3-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,33 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#122231255] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:04,33 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#556203244 for shard member-3-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,33 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/shard-member-3:snapshot-read#-1675202761] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:04,34 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:04,34 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:04,34 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 02:48:04,34 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testModuleShardLeaderMovement is initialized 02:48:04,34 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:04,34 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 02:48:04,34 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 02:48:04,35 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true], state: Idle 02:48:04,35 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 02:48:04,35 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 02:48:04,35 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 02:48:04,35 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,35 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 3 02:48:04,35 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:48:04,35 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-3-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,35 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 02:48:04,35 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@624cf189, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testModuleShardLeaderMovement, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,36 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 02:48:04,36 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 02:48:04,36 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: 229 02:48:04,36 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testModuleShardLeaderMovement, member-3-shard-cars-oper_testModuleShardLeaderMovement), Chunk: 1/1 02:48:04,36 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 02:48:04,36 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 3 02:48:04,36 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 02:48:04,36 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 02:48:04,36 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 02:48:04,44 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]" message is greater than follower's term 0 - updating term 02:48:04,44 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 02:48:04,44 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,44 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 02:48:04,44 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,44 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement to state 02:48:04,44 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-oper_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,45 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 02:48:04,45 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 02:48:04,45 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6c5beb23, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,45 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 02:48:04,45 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done false 02:48:04,46 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:48:04,46 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 02:48:04,46 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 02:48:04,46 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 02:48:04,46 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 02:48:04,46 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 02:48:04,46 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 02:48:04,46 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#1467736574] with name member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 02:48:04,46 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#1467736574] 02:48:04,46 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 557.9 μs 02:48:04,47 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 02:48:04,47 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 02:48:04,47 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 02:48:04,47 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 97.32 μs 02:48:04,47 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#1467736574] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:04,47 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 02:48:04,47 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 02:48:04,48 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 02:48:04,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 02:48:04,48 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 02:48:04,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 02:48:04,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement] 02:48:04,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]] 02:48:04,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,49 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=ec119ffb-362a-4f61-9e0d-16fac8701bfe}, logIndex: 1, isSendImmediate: true 02:48:04,49 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:04,49 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,49 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 02:48:04,49 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:04,49 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,49 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 02:48:04,49 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 02:48:04,50 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,51 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 02:48:04,51 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 02:48:04,51 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:04,51 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 02:48:04,51 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 02:48:04,51 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,51 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 02:48:04,51 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 2 02:48:04,51 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 02:48:04,51 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:04,51 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:04,51 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 02:48:04,51 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,52 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 02:48:04,52 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#556203244 02:48:04,52 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 1 02:48:04,52 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 02:48:04,52 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:04,52 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-oper_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,52 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 02:48:04,52 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,52 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 02:48:04,53 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 02:48:04,53 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Setting last applied to 1 02:48:04,53 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 02:48:04,53 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 02:48:04,53 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 02:48:04,53 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:04,53 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,53 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 02:48:04,53 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:04,53 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 02:48:04,53 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 02:48:04,53 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1018933901] 02:48:04,53 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 02:48:04,53 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,53 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,54 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 02:48:04,54 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 02:48:04,54 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 02:48:04,54 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,54 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:04,54 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 02:48:04,54 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done true 02:48:04,54 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 02:48:04,54 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 02:48:04,54 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#-416294505] 02:48:04,54 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#-416294505] 02:48:04,54 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 02:48:04,55 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 1 02:48:04,55 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:48:04,55 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 1 02:48:04,55 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 02:48:04,55 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 02:48:04,55 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 02:48:04,55 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#799965621] 02:48:04,55 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:04,55 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done false 02:48:04,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038] 02:48:04,55 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038] 02:48:04,56 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 1 02:48:04,56 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:48:04,56 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 1 02:48:04,56 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 02:48:04,56 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 02:48:04,56 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 02:48:04,56 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 02:48:04,56 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1062349602] 02:48:04,56 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:04,56 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-847760943] 02:48:04,56 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-847760943] 02:48:04,57 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#140986663] 02:48:04,57 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:04,57 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1112659353] 02:48:04,57 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1112659353] 02:48:04,58 PM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-1 for shard cars, datastoreType Config 02:48:04,59 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#799965621] 02:48:04,59 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:04,59 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038] 02:48:04,60 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038] 02:48:04,61 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 02:48:04,62 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 02:48:04,63 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#799965621] 02:48:04,63 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:04,63 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038] 02:48:04,63 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038] 02:48:04,68 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1062349602] 02:48:04,68 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:04,68 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-847760943] 02:48:04,68 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-847760943] 02:48:04,68 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#140986663] 02:48:04,68 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:04,68 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1112659353] 02:48:04,69 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1112659353] 02:48:04,69 PM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-2 for shard cars, datastoreType Config 02:48:04,69 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1062349602] 02:48:04,69 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:04,69 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-847760943] 02:48:04,69 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-847760943] 02:48:04,69 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 02:48:04,69 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:04,76 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onRequestLeadership RequestLeadership [requestedFollowerId=member-2-shard-cars-config_testModuleShardLeaderMovement, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/member-2-shard-cars-config_testModuleShardLeaderMovement$e]] 02:48:04,77 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 02:48:04,78 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@48178dbc 02:48:04,78 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@48178dbc 02:48:04,78 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 02:48:04,78 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 02:48:04,78 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,78 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 02:48:04,78 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement] 02:48:04,80 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 02:48:04,80 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement] 02:48:04,80 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 02:48:04,80 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 02:48:04,82 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 02:48:04,82 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 02:48:04,82 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,82 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,82 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 02:48:04,82 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 02:48:04,83 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 02:48:04,84 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 02:48:04,85 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 02:48:04,85 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:04,85 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 2 02:48:04,85 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:04,85 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,85 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 2 02:48:04,85 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:04,85 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 02:48:04,85 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 02:48:04,86 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 02:48:04,86 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 02:48:04,86 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:04,86 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:04,87 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 02:48:04,87 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 02:48:04,87 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 02:48:04,87 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 02:48:04,87 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 2 in "RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:48:04,87 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 2 in "RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than leader's term 1 - switching to Follower 02:48:04,87 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=null 02:48:04,87 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=null 02:48:04,87 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 02:48:04,87 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 1, lastTerm: 1 02:48:04,87 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:04,87 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 1, lastTerm: 1 02:48:04,87 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:04,87 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 02:48:04,87 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 02:48:04,87 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 2 02:48:04,87 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:04,87 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 02:48:04,87 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 02:48:04,88 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 02:48:04,88 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 02:48:04,88 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 02:48:04,88 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 02:48:04,89 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 02:48:04,89 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,89 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,89 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 02:48:04,89 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 02:48:04,89 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 02:48:04,89 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 02:48:04,89 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:04,89 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 02:48:04,89 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:04,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-2-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:04,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-2-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:04,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:04,90 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 02:48:04,90 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-2-shard-cars-config_testModuleShardLeaderMovement in 12.90 ms 02:48:04,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Leadership transferred successfully to member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:04,91 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:04,91 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:04,91 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 02:48:04,92 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:04,92 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:04,92 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 02:48:04,92 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,92 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 02:48:04,92 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,93 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 02:48:04,93 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 02:48:04,98 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 02:48:04,98 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1062349602] 02:48:04,98 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:04,98 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-847760943] 02:48:04,98 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-847760943] 02:48:04,99 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#799965621] 02:48:04,99 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:04,99 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038] 02:48:04,100 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-147252038] 02:48:04,100 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#140986663] 02:48:04,100 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:04,100 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1112659353] 02:48:04,100 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1112659353] 02:48:04,100 PM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-3 for shard cars, datastoreType Config 02:48:04,100 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#140986663] 02:48:04,101 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:04,101 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1112659353] 02:48:04,101 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1112659353] 02:48:04,101 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 02:48:04,101 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-2-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:04,102 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onRequestLeadership RequestLeadership [requestedFollowerId=member-3-shard-cars-config_testModuleShardLeaderMovement, replyTo=Actor[akka://cluster-test@127.0.0.1:2557/temp/member-3-shard-cars-config_testModuleShardLeaderMovement$e]] 02:48:04,103 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 02:48:04,103 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5ede3efb 02:48:04,103 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5ede3efb 02:48:04,103 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 02:48:04,103 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 02:48:04,103 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,103 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,103 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 02:48:04,104 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement] 02:48:04,104 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 02:48:04,105 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement] 02:48:04,105 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 02:48:04,106 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 02:48:04,106 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 02:48:04,106 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,106 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,106 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 02:48:04,106 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 02:48:04,108 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 02:48:04,108 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 02:48:04,108 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,108 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 3 02:48:04,108 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:04,109 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:04,109 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 3 02:48:04,109 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:04,109 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 02:48:04,109 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 02:48:04,109 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 02:48:04,109 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 02:48:04,110 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:04,110 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 02:48:04,110 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 02:48:04,110 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 3 in "RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than leader's term 2 - switching to Follower 02:48:04,110 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=null 02:48:04,110 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 02:48:04,110 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 3, votedFor: null, lastIndex: 1, lastTerm: 1 02:48:04,110 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,110 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:04,110 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 3 02:48:04,110 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:04,110 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 02:48:04,110 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 02:48:04,110 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 02:48:04,110 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 02:48:04,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:04,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 02:48:04,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 02:48:04,111 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 3 in "RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than follower's term 2 - updating term 02:48:04,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=null 02:48:04,111 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:48:04,111 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:48:04,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 3, votedFor: null, lastIndex: 1, lastTerm: 1 02:48:04,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,111 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 02:48:04,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:04,111 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,111 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,111 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:48:04,111 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 02:48:04,111 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 02:48:04,112 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 02:48:04,112 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:04,112 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 02:48:04,112 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:04,112 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-3-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,113 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,113 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-3-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,113 PM [cluster-test-shard-dispatcher-25] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-3-shard-cars-config_testModuleShardLeaderMovement in 10.03 ms 02:48:04,113 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:04,113 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Leadership transferred successfully to member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,113 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 02:48:04,113 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:04,113 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:04,113 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:04,113 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 02:48:04,114 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 02:48:04,114 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,114 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 02:48:04,114 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,115 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 02:48:04,115 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#140986663] 02:48:04,115 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:04,115 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 02:48:04,115 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1112659353] 02:48:04,115 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1112659353] 02:48:04,116 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 02:48:04,116 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 02:48:04,116 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,116 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 02:48:04,116 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onShutDown 02:48:04,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 02:48:04,117 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 02:48:04,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2aebb80e 02:48:04,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2aebb80e 02:48:04,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 02:48:04,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 02:48:04,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,118 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement] 02:48:04,118 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 02:48:04,118 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement] 02:48:04,118 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 02:48:04,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 02:48:04,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 02:48:04,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 02:48:04,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 02:48:04,121 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 02:48:04,121 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 02:48:04,121 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:04,121 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 4 02:48:04,121 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,121 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:04,121 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 4 02:48:04,121 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:04,121 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 02:48:04,121 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 02:48:04,121 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 02:48:04,121 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 02:48:04,122 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:04,122 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:04,122 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 02:48:04,122 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 02:48:04,122 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 02:48:04,122 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 02:48:04,122 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than follower's term 3 - updating term 02:48:04,122 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than leader's term 3 - switching to Follower 02:48:04,122 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=null 02:48:04,122 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=null 02:48:04,122 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 02:48:04,122 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 1, lastTerm: 1 02:48:04,122 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:04,122 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 1, lastTerm: 1 02:48:04,122 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:48:04,122 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:04,122 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:48:04,123 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 4 02:48:04,123 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 02:48:04,123 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 02:48:04,123 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 02:48:04,123 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 02:48:04,123 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 02:48:04,124 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 02:48:04,124 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 02:48:04,124 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 02:48:04,124 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,124 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,124 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 02:48:04,124 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 02:48:04,124 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 02:48:04,124 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 02:48:04,124 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:04,124 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 02:48:04,124 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:04,125 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:04,125 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 02:48:04,125 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:04,125 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:04,125 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-1-shard-cars-config_testModuleShardLeaderMovement in 8.669 ms 02:48:04,125 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:04,125 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer succeeded - sending PoisonPill 02:48:04,126 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:04,126 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 02:48:04,126 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 02:48:04,126 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 02:48:04,126 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testModuleShardLeaderMovement 02:48:04,126 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 02:48:04,126 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 02:48:04,126 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,127 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 02:48:04,127 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 02:48:04,127 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:04,127 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 02:48:04,127 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 02:48:04,127 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 02:48:04,128 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,128 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 02:48:04,128 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 02:48:04,128 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,128 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 02:48:04,128 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 02:48:04,129 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 02:48:04,130 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:04,130 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:04,137 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:04,138 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:04,138 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:04,138 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:04,141 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:48:04,156 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 02:48:04,157 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:04,158 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 02:48:04,158 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onShutDown 02:48:04,158 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testModuleShardLeaderMovement 02:48:04,158 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 02:48:04,159 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 02:48:04,159 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 02:48:04,160 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 02:48:04,160 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,160 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 02:48:04,160 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 02:48:04,163 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testModuleShardLeaderMovement 02:48:04,163 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 02:48:04,163 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 02:48:04,164 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 02:48:04,165 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:04,165 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:04,654 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 02:48:05,144 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-4044410884997460478] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:48:05,184 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [6647913976749233398] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:48:05,186 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:05,186 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:05,193 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:48:05.196] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:05,202 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 02:48:05,202 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:05,202 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testModuleShardLeaderMovement 02:48:05,202 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 02:48:05,202 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onShutDown 02:48:05,203 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 02:48:05,203 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 02:48:05,203 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3249f0b5 02:48:05,203 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3249f0b5 02:48:05,203 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 02:48:05,203 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onShutDown 02:48:05,203 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@68901670 02:48:05,203 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@68901670 02:48:05,203 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-people-config_testModuleShardLeaderMovement 02:48:05,203 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 02:48:05,203 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 02:48:05,204 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:05,204 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:09,761 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 02:48:09,761 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2559, Up)]. 02:48:09,761 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:48:09,761 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:09,761 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:09,761 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:09,761 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:11,491 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:48:12,949 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:48:14,365 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer aborted 02:48:14,365 PM [cluster-test-shard-dispatcher-30] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Failed to transfer leadership in 9.162 s 02:48:14,365 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In unpauseLeader 02:48:14,365 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer failed - sending PoisonPill 02:48:14,365 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer expired 02:48:14,366 PM [cluster-test-shard-dispatcher-30] [WARN] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: At least 1 followers need to be active, Switching member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 02:48:14,367 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 02:48:14,367 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:14,367 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:14,367 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 02:48:14,367 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 02:48:14,367 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 02:48:14,367 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 02:48:14,368 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testModuleShardLeaderMovement 02:48:14,368 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 02:48:14,369 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:48:14,369 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:48:14,369 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 02:48:14,370 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:14,370 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testModuleShardLeaderMovement 02:48:14,370 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 02:48:14,370 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 02:48:14,370 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating leader transfer 02:48:14,371 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onShutDown 02:48:14,371 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@7aa45099 02:48:14,371 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 02:48:14,371 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@7aa45099 02:48:14,371 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7d18e9eb 02:48:14,371 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7d18e9eb 02:48:14,371 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 02:48:14,371 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 02:48:14,371 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:14,371 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:14,371 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testModuleShardLeaderMovement 02:48:14,371 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 02:48:18,54 PM [cluster-test-shard-dispatcher-29] [WARN] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 02:48:18,54 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testModuleShardLeaderMovement, member-2-shard-cars-oper_testModuleShardLeaderMovement] 02:48:18,54 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:18,54 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:18,54 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 02:48:18,54 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer aborted 02:48:18,55 PM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: Failed to transfer leadership in 3.684 s 02:48:18,55 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In unpauseLeader 02:48:18,55 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer failed - sending PoisonPill 02:48:18,55 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 02:48:18,55 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 02:48:18,55 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 02:48:18,56 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 02:48:18,56 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 02:48:18,56 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:48:18,56 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:48:18,59 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:18,59 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:18,63 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:48:18.066] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:18,83 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:48:18,96 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3964384000589362794] 02:48:18,97 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:48:18,100 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:48:18,100 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:48:18,103 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:18,103 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:18,120 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:48:18,120 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:48:18,120 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:48:18,120 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:18,122 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:18,124 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 02:48:18,131 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#1983038479] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,132 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:48:18,135 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:48:18,135 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardReplica 02:48:18,136 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardReplica 02:48:18,136 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:18,137 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:18,137 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:18,141 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 02:48:18,145 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@43b5b646 02:48:18,149 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} started 02:48:18,150 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:18,150 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:18,150 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardReplica 02:48:18,151 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:18,151 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:18,151 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica on shard member-1-shard-cars-config_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1000088481] 02:48:18,151 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardReplica 02:48:18,152 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:18,152 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:18,152 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:18,152 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica on shard member-1-shard-people-config_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#-373509110] 02:48:18,153 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveShardReplica, persistent : true 02:48:18,153 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1000088481] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,154 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#337834485 created and ready for shard:member-1-shard-cars-config_testRemoveShardReplica 02:48:18,154 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/shard-member-1:snapshot-read#823876265] 02:48:18,154 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#337834485] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,154 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveShardReplica with journal batch size 1 02:48:18,155 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 02:48:18,156 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveShardReplica, persistent : true 02:48:18,156 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#-1616924378 created and ready for shard:member-1-shard-people-config_testRemoveShardReplica 02:48:18,156 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/shard-member-1:snapshot-read#1512012090] 02:48:18,157 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveShardReplica with journal batch size 1 02:48:18,157 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:18,157 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:18,158 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/shard-member-1:snapshot-read#823876265] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,158 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 02:48:18,158 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardReplica is initialized 02:48:18,158 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:18,158 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:18,159 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:48:18,159 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 02:48:18,159 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardReplica is initialized 02:48:18,161 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,161 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardReplica is no longer the leader 02:48:18,161 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 02:48:18,161 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 02:48:18,161 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from null to Follower 02:48:18,161 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#-373509110] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardReplica is no longer the leader 02:48:18,162 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from null to Follower 02:48:18,162 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 02:48:18,162 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from null to Follower 02:48:18,162 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:48:18,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardReplica 02:48:18,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardReplica 02:48:18,163 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:18,163 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 02:48:18,163 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 02:48:18,165 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1848110034] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,165 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#-1616924378] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,165 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/shard-member-1:snapshot-read#1512012090] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,166 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 02:48:18,167 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6ae470c0 02:48:18,167 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 02:48:18,171 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:18,172 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardReplica 02:48:18,172 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 02:48:18,172 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 02:48:18,172 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-1-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#340007633] 02:48:18,172 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardReplica 02:48:18,172 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveShardReplica, persistent : true 02:48:18,173 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#340007633] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,173 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 02:48:18,173 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 02:48:18,173 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-1-shard-people-oper_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#2059337931] 02:48:18,173 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveShardReplica, persistent : true 02:48:18,173 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#-1371709296 created and ready for shard:member-1-shard-cars-oper_testRemoveShardReplica 02:48:18,173 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/shard-member-1:snapshot-read#-1556704921] 02:48:18,173 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#2059337931] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,174 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/shard-member-1:snapshot-read#-532102571] 02:48:18,174 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveShardReplica with journal batch size 1 02:48:18,177 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:18,177 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:18,177 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 02:48:18,177 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardReplica is initialized 02:48:18,177 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,177 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardReplica is no longer the leader 02:48:18,177 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveShardReplica with journal batch size 1 02:48:18,177 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#1058452520 created and ready for shard:member-1-shard-people-oper_testRemoveShardReplica 02:48:18,178 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 02:48:18,178 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 02:48:18,178 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from null to Follower 02:48:18,178 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:18,178 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:18,178 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 02:48:18,178 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardReplica is initialized 02:48:18,179 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,179 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardReplica is no longer the leader 02:48:18,179 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 02:48:18,179 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 02:48:18,179 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from null to Follower 02:48:18,191 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:48:18,203 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2458511130577356900] 02:48:18,204 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:48:18,206 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:18,206 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:48:18,217 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:18,218 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:18,220 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:18,220 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:18,223 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 02:48:18,229 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1941139649] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,230 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 02:48:18,231 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 02:48:18,231 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 02:48:18,232 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 02:48:18,234 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 02:48:18,235 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7ad3bce8 02:48:18,235 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} started 02:48:18,236 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:18,236 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 02:48:18,236 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:18,237 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 02:48:18,238 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardReplica, persistent : true 02:48:18,249 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:18,255 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 02:48:18,257 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardReplica with journal batch size 1 02:48:18,257 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#2136009663 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 02:48:18,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#1519278677] 02:48:18,257 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1535035387] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,257 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#2136009663] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,257 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardReplica, persistent : true 02:48:18,258 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#1519278677] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,258 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#79276964 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 02:48:18,271 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 02:48:18,276 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 02:48:18,276 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 02:48:18,277 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 02:48:18,278 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:18,278 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:18,278 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 02:48:18,278 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 02:48:18,278 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,278 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardReplica is no longer the leader 02:48:18,278 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 02:48:18,279 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#-1052002674] 02:48:18,279 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardReplica with journal batch size 1 02:48:18,279 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 02:48:18,280 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3b07d40f 02:48:18,280 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 02:48:18,285 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:18,285 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:18,285 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:18,285 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 02:48:18,285 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,285 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveShardReplica is no longer the leader 02:48:18,285 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 02:48:18,285 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 02:48:18,285 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 02:48:18,285 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 02:48:18,286 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#2058132502] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,286 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#79276964] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,286 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 02:48:18,286 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1279579811] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,286 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#-1052002674] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,286 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 02:48:18,286 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardReplica, persistent : true 02:48:18,286 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardReplica, persistent : true 02:48:18,287 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-2102817617 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 02:48:18,287 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/shard-member-2:snapshot-read#-1290047765] 02:48:18,287 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardReplica with journal batch size 1 02:48:18,287 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardReplica with journal batch size 1 02:48:18,287 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#960253331 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 02:48:18,287 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/shard-member-2:snapshot-read#1773716933] 02:48:18,288 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#-739937662] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,288 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#553559395] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,288 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:18,288 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:18,288 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:18,289 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:18,289 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,289 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,289 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveShardReplica is no longer the leader 02:48:18,289 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveShardReplica is no longer the leader 02:48:18,289 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 02:48:18,289 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 02:48:18,289 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 02:48:18,289 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 02:48:18,289 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 02:48:18,289 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 02:48:18,289 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 02:48:18,289 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 02:48:18,289 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 02:48:18,289 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 02:48:18,328 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 02:48:18,329 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 02:48:18,349 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:48:18,358 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:48:18,362 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:18,362 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:18,362 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 02:48:18,362 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:18,362 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 02:48:18,362 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:18,362 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:18,362 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:18,362 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 02:48:18,362 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:48:18,362 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 02:48:18,362 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 02:48:18,362 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 02:48:18,362 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 02:48:18,362 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 02:48:18,368 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7761039486388585930] 02:48:18,369 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 02:48:18,372 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:18,372 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 02:48:18,374 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:18,374 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:18,386 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:18,388 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:18,395 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 02:48:18,401 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#2071202544] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,402 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 02:48:18,403 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 02:48:18,404 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardReplica 02:48:18,404 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardReplica 02:48:18,407 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 02:48:18,408 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4892d85e 02:48:18,408 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} started 02:48:18,409 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:18,410 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:18,410 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardReplica 02:48:18,410 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardReplica 02:48:18,411 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:18,411 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveShardReplica, persistent : true 02:48:18,412 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveShardReplica with journal batch size 1 02:48:18,413 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveShardReplica, persistent : true 02:48:18,413 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveShardReplica with journal batch size 1 02:48:18,413 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 02:48:18,415 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 02:48:18,415 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#-1585255379 created and ready for shard:member-3-shard-cars-config_testRemoveShardReplica 02:48:18,417 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1963137786] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,417 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#-1539470799 created and ready for shard:member-3-shard-people-config_testRemoveShardReplica 02:48:18,417 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/shard-member-3:snapshot-read#-1237630904] 02:48:18,419 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/shard-member-3:snapshot-read#-1819319742] 02:48:18,421 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica#521447080] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,421 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1780520147] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,421 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#-1585255379] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,421 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#-1539470799] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,421 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/shard-member-3:snapshot-read#-1237630904] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,421 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/shard-member-3:snapshot-read#-1819319742] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,422 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:18,422 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:18,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:18,422 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:18,423 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,423 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testRemoveShardReplica is no longer the leader 02:48:18,423 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , received role change from null to Follower 02:48:18,423 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 02:48:18,423 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardReplica 02:48:18,423 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardReplica 02:48:18,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 02:48:18,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardReplica is initialized 02:48:18,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 02:48:18,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardReplica is initialized 02:48:18,424 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 02:48:18,424 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardReplica from null to Follower 02:48:18,425 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,425 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveShardReplica is no longer the leader 02:48:18,425 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 02:48:18,425 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 02:48:18,425 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 02:48:18,426 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardReplica from null to Follower 02:48:18,426 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@357052ab 02:48:18,426 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 02:48:18,427 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:18,427 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardReplica 02:48:18,427 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 02:48:18,428 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardReplica 02:48:18,428 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveShardReplica, persistent : true 02:48:18,428 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#1513870564] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,429 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/shard-member-3:snapshot-read#245578283] 02:48:18,429 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#-1636178378 created and ready for shard:member-3-shard-cars-oper_testRemoveShardReplica 02:48:18,429 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveShardReplica with journal batch size 1 02:48:18,429 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#-1636178378] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:18,429 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveShardReplica, persistent : true 02:48:18,430 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#-1051291689 created and ready for shard:member-3-shard-people-oper_testRemoveShardReplica 02:48:18,430 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/shard-member-3:snapshot-read#-595505286] 02:48:18,431 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:18,431 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveShardReplica with journal batch size 1 02:48:18,431 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:18,431 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 02:48:18,431 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,431 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardReplica is initialized 02:48:18,431 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testRemoveShardReplica is no longer the leader 02:48:18,431 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 02:48:18,431 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 02:48:18,432 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardReplica from null to Follower 02:48:18,432 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:18,432 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:18,432 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 02:48:18,432 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardReplica is initialized 02:48:18,432 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,432 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testRemoveShardReplica is no longer the leader 02:48:18,433 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 02:48:18,433 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 02:48:18,433 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardReplica from null to Follower 02:48:18,450 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 02:48:18,453 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:48:18,454 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:18,454 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 02:48:18,454 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 02:48:18,454 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 02:48:18,454 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 02:48:18,454 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 02:48:18,454 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 02:48:18,454 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:18,454 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:18,454 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 02:48:18,454 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:18,454 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 02:48:18,454 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:18,454 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:18,499 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:48:18,499 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:48:18,499 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 02:48:18,499 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 02:48:18,499 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testRemoveShardReplica, member-2-shard-cars-config_testRemoveShardReplica] 02:48:18,499 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 02:48:18,499 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-config_testRemoveShardReplica 02:48:18,499 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-people-config_testRemoveShardReplica 02:48:18,499 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 1 02:48:18,499 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 1 02:48:18,499 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:18,499 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:18,499 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,499 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardReplica is no longer the leader 02:48:18,500 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,500 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardReplica is no longer the leader 02:48:18,500 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Follower to Candidate 02:48:18,500 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Follower to Candidate 02:48:18,500 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Follower to Candidate 02:48:18,500 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Follower to Candidate 02:48:18,568 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:48:18,569 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 02:48:18,569 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 02:48:18,569 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-people-oper_testRemoveShardReplica 02:48:18,569 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 1 02:48:18,569 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:18,569 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,569 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardReplica is no longer the leader 02:48:18,569 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Follower to Candidate 02:48:18,569 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Follower to Candidate 02:48:18,599 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:48:18,599 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 02:48:18,599 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 02:48:18,599 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 02:48:18,599 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 1 02:48:18,599 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:18,599 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:18,599 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardReplica is no longer the leader 02:48:18,600 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Follower to Candidate 02:48:18,600 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Follower to Candidate 02:48:18,899 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 02:48:18,899 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 02:48:18,899 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-cars-config_testRemoveShardReplica 02:48:18,899 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-people-config_testRemoveShardReplica 02:48:18,899 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 2 02:48:18,899 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 2 02:48:18,949 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 02:48:18,949 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 02:48:18,949 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 2 02:48:18,968 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 02:48:18,968 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-people-oper_testRemoveShardReplica 02:48:18,969 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 2 02:48:19,121 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 02:48:19,121 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:48:19,122 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:19,122 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:19,122 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:19,122 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:19,122 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 02:48:19,122 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 02:48:19,123 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 02:48:19,123 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 02:48:19,123 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 02:48:19,123 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:19,132 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:19,132 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:19,132 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:19,132 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 02:48:19,132 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 02:48:19,239 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 02:48:19,239 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 02:48:19,239 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 3 02:48:19,239 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testRemoveShardReplica 02:48:19,240 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testRemoveShardReplica 02:48:19,242 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:19,242 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:19,242 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Cluster state: [] 02:48:19,242 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Cluster state: [] 02:48:19,242 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:19,242 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:19,242 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:19,242 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:19,242 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 02:48:19,242 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 02:48:19,242 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:19,242 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 02:48:19,243 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:19,243 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:19,243 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 02:48:19,243 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:19,245 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:48:19,245 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:48:19,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 02:48:19,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:19,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:19,246 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:48:19,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:19,246 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: became leader with frontend state for [] 02:48:19,246 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Candidate to Leader 02:48:19,246 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 02:48:19,246 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Candidate to Leader 02:48:19,250 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:19,250 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 02:48:19,250 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 02:48:19,251 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:19,251 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 02:48:19,251 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done false 02:48:19,251 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 02:48:19,251 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:19,252 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 02:48:19,253 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:19,254 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:19,254 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:19,254 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:19,254 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:19,254 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 02:48:19,254 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 02:48:19,268 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 02:48:19,268 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 02:48:19,268 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 3 02:48:19,268 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testRemoveShardReplica 02:48:19,269 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testRemoveShardReplica 02:48:19,270 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:19,270 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:19,270 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Cluster state: [] 02:48:19,270 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Cluster state: [] 02:48:19,270 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:19,270 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:19,270 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:19,270 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:19,270 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 02:48:19,270 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 02:48:19,270 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:19,270 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:19,270 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 02:48:19,270 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 02:48:19,270 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:19,270 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:19,272 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:48:19,272 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:48:19,272 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testRemoveShardReplica, member-2-shard-cars-config_testRemoveShardReplica] 02:48:19,272 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:19,272 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:19,272 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:48:19,272 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:19,273 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: became leader with frontend state for [] 02:48:19,273 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Candidate to Leader 02:48:19,273 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 02:48:19,273 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Candidate to Leader 02:48:19,273 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 02:48:19,273 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 02:48:19,273 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 02:48:19,274 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:19,274 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 02:48:19,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:19,274 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardReplica is ready 02:48:19,274 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 02:48:19,274 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardReplica status sync done false 02:48:19,274 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 02:48:19,274 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#1983038479] 02:48:19,274 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 02:48:19,274 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 02:48:19,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:19,275 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1000088481] 02:48:19,275 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1000088481] 02:48:19,275 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1941139649] 02:48:19,275 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 02:48:19,276 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:19,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:19,276 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1535035387] 02:48:19,276 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1535035387] 02:48:19,276 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#2071202544] 02:48:19,276 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 02:48:19,276 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:19,277 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1963137786] 02:48:19,277 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:19,277 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1963137786] 02:48:19,279 PM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-3, datastoreType Config 02:48:19,279 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 02:48:19,281 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:19,281 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, primaryVersion=13] 02:48:19,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica for shard member-3-shard-cars-config_testRemoveShardReplica 02:48:19,285 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-config_testRemoveShardReplica'}, state: Idle 02:48:19,287 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]] 02:48:19,287 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 02:48:19,287 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 02:48:19,288 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=9eb78b2c-9313-46b4-b10d-2b3a1f3143f8}, logIndex: 0, isSendImmediate: true 02:48:19,288 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:19,288 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveShardReplica 02:48:19,288 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]]]] 02:48:19,288 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-config_testRemoveShardReplica'} 02:48:19,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]]]] 02:48:19,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Number of entries to be appended = 1 02:48:19,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:19,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 02:48:19,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 02:48:19,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, votingState=VOTING]] 02:48:19,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Log size is now 1 02:48:19,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:19,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 02:48:19,291 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 02:48:19,291 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 02:48:19,292 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 02:48:19,292 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 02:48:19,292 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 02:48:19,292 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardReplica updated: matchIndex: 0, nextIndex: 1 02:48:19,292 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:48:19,292 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Setting last applied to 0 02:48:19,292 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 02:48:19,292 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 02:48:19,292 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:19,292 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:19,293 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=0 02:48:19,293 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#1983038479] 02:48:19,293 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:19,294 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1000088481] 02:48:19,294 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1000088481] 02:48:19,294 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1941139649] 02:48:19,294 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:19,294 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1535035387] 02:48:19,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Commit index set to 0 02:48:19,295 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1535035387] 02:48:19,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:19,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Setting last applied to 0 02:48:19,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 02:48:19,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 02:48:19,295 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=0 02:48:19,295 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#2071202544] 02:48:19,296 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1963137786] with 27000 ms timeout 02:48:19,296 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onShutDown 02:48:19,296 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveShardReplica 02:48:19,296 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:19,298 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#1963137786] 02:48:19,298 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 02:48:19,299 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [people] 02:48:19,299 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:19,299 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [people] 02:48:19,299 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 02:48:19,300 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:19,300 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 02:48:19,300 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node [akka://cluster-test@127.0.0.1:2559] as [Down] 02:48:19,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardReplica 02:48:19,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 02:48:19,303 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 02:48:19,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onShutDown 02:48:19,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onShutDown 02:48:19,303 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardReplica 02:48:19,303 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardReplica 02:48:19,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:19,303 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:19,303 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 02:48:19,303 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 02:48:19,304 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 02:48:19,304 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Received gossip where this member has been downed, from [akka://cluster-test@127.0.0.1:2558] 02:48:19,304 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 02:48:19,304 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 02:48:19,304 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 02:48:19,304 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onShutDown 02:48:19,304 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onShutDown 02:48:19,305 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardReplica 02:48:19,305 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:19,305 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardReplica 02:48:19,305 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:19,305 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 02:48:19,305 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 02:48:19,305 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Node has been marked as DOWN. Shutting down myself 02:48:19,306 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:19,306 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:19,307 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Shutting down... 02:48:19,307 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Successfully shut down 02:48:19,313 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:19,313 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:19,313 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:19,313 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:19,315 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:48:19,343 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:48:19,353 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5072264002092269038] 02:48:19,354 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:48:19,357 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Registered cluster JMX MBean [akka:type=Cluster] 02:48:19,357 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:48:19,358 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 02:48:19,358 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 02:48:19,358 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardReplica 02:48:19,358 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 02:48:19,358 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 3 02:48:19,358 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 3 02:48:19,358 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testRemoveShardReplica 02:48:19,358 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testRemoveShardReplica 02:48:19,359 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testRemoveShardReplica 02:48:19,359 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testRemoveShardReplica 02:48:19,360 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:19,361 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:19,362 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:19,362 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:19,362 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Cluster state: [] 02:48:19,362 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:19,362 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Cluster state: [] 02:48:19,362 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:19,362 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 02:48:19,362 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:19,362 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 02:48:19,362 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:19,363 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:48:19,363 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:48:19,363 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 02:48:19,363 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:19,363 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:19,363 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:48:19,363 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:19,363 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: became leader with frontend state for [] 02:48:19,363 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Candidate to Leader 02:48:19,363 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 02:48:19,364 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Candidate to Leader 02:48:19,362 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:19,364 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:19,364 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 02:48:19,364 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:19,364 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardReplica 02:48:19,364 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:19,365 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:48:19,365 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:48:19,365 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 02:48:19,365 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:19,366 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:19,366 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:48:19,366 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:19,366 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: became leader with frontend state for [] 02:48:19,366 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Candidate to Leader 02:48:19,366 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 02:48:19,366 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Candidate to Leader 02:48:19,366 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 02:48:19,367 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 02:48:19,367 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 02:48:19,367 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 02:48:19,367 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done false 02:48:19,367 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 02:48:19,368 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardReplica is ready 02:48:19,368 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done false 02:48:19,368 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 02:48:19,368 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 02:48:19,368 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:19,368 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:19,380 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:19,381 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:19,383 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 02:48:19,390 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 02:48:19,401 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2127904086] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:19,403 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 02:48:19,405 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 02:48:19,405 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 02:48:19,406 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 02:48:19,411 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@23994119 02:48:19,411 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} started 02:48:19,412 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:19,414 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:19,415 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:19,415 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 02:48:19,416 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 02:48:19,416 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardReplica, persistent : true 02:48:19,416 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1988629690] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:19,417 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#903077894 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 02:48:19,417 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardReplica with journal batch size 1 02:48:19,417 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#903077894] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:19,417 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#1637439459] 02:48:19,418 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#778235767] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:19,419 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:19,420 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 02:48:19,420 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:19,420 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 02:48:19,420 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:19,420 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Received ReplicatedLogEntry for recovery: index: 0, size: 221 02:48:19,420 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveShardReplica, address=null, votingState=VOTING]] 02:48:19,420 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 02:48:19,420 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Received apply journal entries for recovery, applying to state: 0 to 0 02:48:19,420 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 02:48:19,420 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:19,420 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed in 202.8 μs - Switching actor to Follower - last log index = 0, last log term = 3, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:19,422 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 02:48:19,426 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 02:48:19,426 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 02:48:19,429 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardReplica, persistent : true 02:48:19,430 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#905165610] 02:48:19,430 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardReplica with journal batch size 1 02:48:19,431 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-694522165 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 02:48:19,431 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:19,431 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 02:48:19,431 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:19,431 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 02:48:19,432 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:19,432 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:19,432 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 02:48:19,432 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 02:48:19,432 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 02:48:19,434 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#1637439459] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:19,434 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-1422268801] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:19,434 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#905165610] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:19,434 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-694522165] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:19,434 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:19,434 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardReplica is no longer the leader 02:48:19,434 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 02:48:19,434 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 02:48:19,435 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 02:48:19,435 PM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:19,435 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveShardReplica is no longer the leader 02:48:19,435 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 02:48:19,435 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 02:48:19,435 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 02:48:19,437 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 02:48:19,438 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 02:48:19,439 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 02:48:19,440 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@652854b8 02:48:19,440 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} started 02:48:19,441 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 02:48:19,445 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:19,445 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 02:48:19,445 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 02:48:19,446 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardReplica, persistent : true 02:48:19,447 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardReplica with journal batch size 1 02:48:19,447 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/shard-member-2:snapshot-read#-2020818124] 02:48:19,447 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#-605645454] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:19,447 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-1370941093 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 02:48:19,447 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/shard-member-2:snapshot-read#-2020818124] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:19,447 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardReplica, persistent : true 02:48:19,448 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#-770461439 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 02:48:19,448 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/shard-member-2:snapshot-read#-183399211] 02:48:19,448 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:19,448 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:19,449 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 02:48:19,449 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 02:48:19,449 PM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:19,449 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveShardReplica is no longer the leader 02:48:19,449 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 02:48:19,449 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 02:48:19,449 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardReplica with journal batch size 1 02:48:19,449 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 02:48:19,450 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:19,450 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:19,450 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 02:48:19,450 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:19,450 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 02:48:19,450 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveShardReplica is no longer the leader 02:48:19,450 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 02:48:19,450 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 02:48:19,450 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 02:48:19,562 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done true 02:48:19,563 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 310, lastApplied : -1, commitIndex : -1 02:48:19,681 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done true 02:48:19,682 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done true 02:48:19,682 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 313, lastApplied : -1, commitIndex : -1 02:48:19,683 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 314, lastApplied : -1, commitIndex : -1 02:48:19,882 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:20,3 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:48:20,4 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:48:20,139 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 02:48:20,203 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:20,320 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2458511130577356900] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:48:20,322 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [-2458511130577356900] but current is [-5072264002092269038]. Reason: ActorSystem terminated 02:48:20,324 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:20,324 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:20,370 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 02:48:20,378 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:20,378 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Cluster state: [] 02:48:20,378 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:20,378 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:20,378 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 02:48:20,378 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:20,378 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardReplica 02:48:20,378 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:20,379 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:20,379 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Cluster state: [] 02:48:20,379 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:20,379 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:20,379 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 02:48:20,379 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:20,379 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 02:48:20,379 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:20,380 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardReplica peerAddress = null 02:48:20,380 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:20,380 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveShardReplica peerAddress = null 02:48:20,380 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:20,381 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 02:48:20,381 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 02:48:20,381 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 02:48:20,381 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveShardReplica peerAddress = null 02:48:20,381 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:20,381 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 02:48:20,381 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done true 02:48:20,381 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 02:48:20,381 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 02:48:20,381 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardReplica peerAddress = null 02:48:20,381 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:20,381 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 02:48:20,381 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 02:48:20,381 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 02:48:20,383 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 02:48:20,383 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:20,383 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 02:48:20,383 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 1132, lastApplied : -1, commitIndex : -1 02:48:20,383 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 02:48:20,384 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:20,384 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done true 02:48:20,384 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 1085, lastApplied : 0, commitIndex : 0 02:48:20,384 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done true 02:48:20,386 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:20,386 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:20,432 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#1606320899]] intended for system uid [-2458511130577356900], self uid is [-5072264002092269038] 02:48:20,522 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:20,643 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:20,644 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:20,841 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:20,841 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:20,843 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:20,843 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 458, lastApplied : -1, commitIndex : -1 02:48:20,873 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 486, lastApplied : 0, commitIndex : 0 02:48:20,962 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 02:48:20,962 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 02:48:20,962 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 02:48:20,962 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 02:48:20,963 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 579, lastApplied : -1, commitIndex : -1 02:48:20,964 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:20,964 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 578, lastApplied : -1, commitIndex : -1 02:48:20,964 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:21,162 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:21,164 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:21,191 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : 0, commitIndex : 0 02:48:21,283 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:21,283 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:21,283 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:21,283 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:21,402 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 02:48:21,483 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:21,483 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:21,512 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 02:48:21,602 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:21,602 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:21,603 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:21,603 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:21,802 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:21,802 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:21,831 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : 0, commitIndex : 0 02:48:21,923 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:21,923 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:21,924 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:48:21,924 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:48:22,122 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:22,122 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:22,151 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#1139029392]] intended for system uid [-2458511130577356900], self uid is [-5072264002092269038] 02:48:22,153 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 02:48:22,243 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:22,243 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:22,243 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:22,243 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:22,422 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 02:48:22,442 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:22,442 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:22,472 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 02:48:22,563 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:22,563 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:22,563 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:22,563 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:22,763 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:22,763 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:22,791 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 02:48:22,883 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:22,883 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:22,883 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:22,884 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:23,83 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:23,83 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:23,112 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 02:48:23,202 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:23,203 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:23,204 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:23,204 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:23,403 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:23,404 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:23,431 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 02:48:23,441 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 02:48:23,522 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 02:48:23,522 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 02:48:23,522 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:23,522 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:23,723 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:23,724 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:23,751 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 02:48:23,841 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:23,842 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:23,842 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:23,842 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:24,41 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 311, lastApplied : -1, commitIndex : -1 02:48:24,42 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:24,71 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 02:48:24,163 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:48:24,163 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:48:24,163 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:24,164 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:48:24,359 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 02:48:24,359 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 02:48:24,359 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:24,359 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:24,359 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 02:48:24,359 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:24,359 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 02:48:24,360 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 02:48:24,362 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:24,362 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - IsolatedLeader member-1-shard-people-config_testRemoveShardReplica switching from IsolatedLeader to Leader 02:48:24,362 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 02:48:24,363 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:24,363 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:24,363 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 02:48:24,363 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:24,363 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 02:48:24,363 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:24,363 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from IsolatedLeader to Leader 02:48:24,363 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 02:48:24,389 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 02:48:24,389 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardReplica] 02:48:24,389 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:24,389 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 02:48:24,389 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:24,390 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 02:48:24,390 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 02:48:24,392 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 02:48:24,392 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - IsolatedLeader member-1-shard-cars-config_testRemoveShardReplica switching from IsolatedLeader to Leader 02:48:24,392 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardReplica] 02:48:24,392 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:24,392 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 02:48:24,392 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:24,392 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 02:48:24,392 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from IsolatedLeader to Leader 02:48:24,392 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 02:48:24,442 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#1606320899]] intended for system uid [-2458511130577356900], self uid is [-5072264002092269038] 02:48:24,461 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 02:48:24,479 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 02:48:24,479 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 02:48:24,479 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 02:48:24,479 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:24,479 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:24,479 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 02:48:24,479 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 02:48:24,479 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:24,479 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:24,479 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 02:48:24,479 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:24,479 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 02:48:24,479 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:24,480 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 02:48:24,480 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 02:48:24,480 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 02:48:24,482 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:24,482 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - IsolatedLeader member-1-shard-people-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 02:48:24,482 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 02:48:24,482 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:24,482 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:24,482 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 02:48:24,482 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:24,482 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 02:48:24,483 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from IsolatedLeader to Leader 02:48:24,483 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:24,483 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - IsolatedLeader member-1-shard-cars-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 02:48:24,483 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 02:48:24,483 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:24,483 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:24,483 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 02:48:24,483 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:24,483 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 02:48:24,483 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from IsolatedLeader to Leader 02:48:24,483 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 02:48:24,484 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:24,485 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:48:24,682 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 02:48:24,683 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 02:48:24,711 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 02:48:24,804 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 02:48:24,804 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 02:48:24,805 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:24,805 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:25,2 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:25,2 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:25,32 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 02:48:25,123 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:25,123 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:25,123 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:25,123 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:25,239 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2559, Down)]. 02:48:25,241 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:25,241 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:25,258 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:25,258 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:25,322 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:25,322 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:25,351 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 02:48:25,442 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:25,443 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:25,443 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:25,443 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:25,480 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 02:48:25,512 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2559] 02:48:25,514 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:25,514 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:25,514 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [-2458511130577356900] but current is [-5072264002092269038]. Reason: Cluster member removed, previous status [Down] 02:48:25,607 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:25,607 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:25,607 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [-2458511130577356900] but current is [-5072264002092269038]. Reason: Cluster member removed, previous status [Down] 02:48:25,641 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:25,641 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:25,671 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 02:48:25,763 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:25,763 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:25,765 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 323, lastApplied : -1, commitIndex : -1 02:48:25,765 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 02:48:25,962 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:48:25,962 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:48:25,992 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 02:48:26,82 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:26,82 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:26,83 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 02:48:26,83 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 02:48:26,281 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:26,282 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:26,310 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 02:48:26,401 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:26,401 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:48:26,402 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:26,402 PM [cluster-test-shard-dispatcher-17] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:48:26,501 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:48:26,506 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:48:26,507 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:26,507 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 02:48:26,507 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 02:48:26,507 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 02:48:26,507 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 02:48:26,507 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 02:48:26,507 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 02:48:26,507 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 02:48:26,507 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:26,507 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 02:48:26,507 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:26,507 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 02:48:26,507 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 02:48:26,507 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 02:48:26,507 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 02:48:26,507 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 02:48:26,507 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 02:48:26,507 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 02:48:26,507 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 02:48:26,507 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 02:48:26,508 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 02:48:26,508 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 02:48:26,508 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 02:48:26,508 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:26,508 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:26,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 02:48:26,508 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:26,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 02:48:26,508 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 02:48:26,508 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 02:48:26,508 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 02:48:26,509 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} is now ready 02:48:26,509 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2127904086] 02:48:26,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:26,510 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1988629690] 02:48:26,510 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1988629690] 02:48:26,511 PM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-2, datastoreType Config 02:48:26,511 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-2}] 02:48:26,511 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:26,512 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1000088481, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@122a386d] 02:48:26,512 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1000088481 for shard member-2-shard-cars-config_testRemoveShardReplica 02:48:26,513 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: onRemoveServer: RemoveServer{serverId='member-2-shard-cars-config_testRemoveShardReplica'}, state: Idle 02:48:26,513 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: New server configuration : [ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]] 02:48:26,513 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 02:48:26,513 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 02:48:26,513 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: Returning OK for operation RemoveServer{serverId='member-2-shard-cars-config_testRemoveShardReplica'} 02:48:26,513 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 02:48:26,514 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 02:48:26,514 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 02:48:26,514 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#1983038479] 02:48:26,514 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:26,514 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1000088481] 02:48:26,514 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1000088481] 02:48:26,514 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2127904086] 02:48:26,514 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 02:48:26,514 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Moving last applied index from 0 to 1 02:48:26,514 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 02:48:26,514 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:26,514 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=1 02:48:26,515 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:26,515 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1988629690] 02:48:26,515 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1988629690] with 27000 ms timeout 02:48:26,516 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2 : Local Shard replica for shard cars has been removed 02:48:26,516 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [people] 02:48:26,516 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onShutDown 02:48:26,516 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardReplica 02:48:26,516 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:26,517 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1988629690] 02:48:26,517 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:26,517 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 02:48:26,565 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2127904086] 02:48:26,565 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:26,565 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Local shard cars not found - shards present: [people] 02:48:26,565 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 02:48:26,566 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 02:48:26,566 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 02:48:26,566 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onShutDown 02:48:26,566 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardReplica 02:48:26,566 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:26,567 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 02:48:26,567 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 02:48:26,567 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} 02:48:26,568 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 02:48:26,568 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 02:48:26,568 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 02:48:26,568 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onShutDown 02:48:26,568 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onShutDown 02:48:26,568 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardReplica 02:48:26,568 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:26,569 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardReplica 02:48:26,569 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:26,569 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 02:48:26,569 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 02:48:26,571 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:26,571 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:26,579 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:26,579 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:26,579 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:26,579 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:26,581 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:48:26,602 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:48:26,602 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 02:48:26,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardReplica 02:48:26,603 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 02:48:26,603 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onShutDown 02:48:26,604 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveShardReplica 02:48:26,604 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:26,604 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 02:48:26,605 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 02:48:26,605 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 02:48:26,605 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardReplica 02:48:26,605 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardReplica 02:48:26,605 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 02:48:26,606 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onShutDown 02:48:26,606 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onShutDown 02:48:26,606 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveShardReplica 02:48:26,606 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveShardReplica 02:48:26,606 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:26,606 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:26,607 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 02:48:26,607 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 02:48:26,608 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:26,608 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:27,552 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:48:27,589 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-5072264002092269038] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:48:27,623 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:27,623 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:27,625 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:48:27.628] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:27,629 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-7761039486388585930] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:48:27,640 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 02:48:27,641 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveShardReplica 02:48:27,641 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardReplica 02:48:27,641 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onShutDown 02:48:27,641 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@36cce8d0 02:48:27,642 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@36cce8d0 02:48:27,642 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 02:48:27,642 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onShutDown 02:48:27,642 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Initiating leader transfer 02:48:27,642 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveShardReplica 02:48:27,642 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:27,642 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@32f36ad1 02:48:27,642 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@32f36ad1 02:48:27,642 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 02:48:27,642 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Attempting to transfer leadership 02:48:27,642 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:27,642 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:27,643 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 02:48:28,199 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: leader transfer aborted 02:48:28,199 PM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: Failed to transfer leadership in 557.3 ms 02:48:28,199 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In unpauseLeader 02:48:28,200 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: leader transfer failed - sending PoisonPill 02:48:28,200 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Leadership transfer expired 02:48:28,200 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveShardReplica 02:48:28,200 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:28,201 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:48:28,201 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:48:28,202 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 02:48:28,202 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardReplica 02:48:28,202 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardReplica 02:48:28,203 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 02:48:28,203 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onShutDown 02:48:28,203 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Initiating leader transfer 02:48:28,203 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onShutDown 02:48:28,203 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Initiating leader transfer 02:48:28,203 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1615aef0 02:48:28,203 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1615aef0 02:48:28,203 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 02:48:28,203 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Attempting to transfer leadership 02:48:28,203 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:28,203 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:28,203 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 02:48:28,203 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@77f1e2eb 02:48:28,203 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@77f1e2eb 02:48:28,203 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 02:48:28,203 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Attempting to transfer leadership 02:48:28,203 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 02:48:28,203 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:28,203 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:28,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: leader transfer aborted 02:48:28,639 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: leader transfer aborted 02:48:28,639 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: Failed to transfer leadership in 436.2 ms 02:48:28,639 PM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: Failed to transfer leadership in 436.0 ms 02:48:28,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In unpauseLeader 02:48:28,639 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In unpauseLeader 02:48:28,639 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: leader transfer failed - sending PoisonPill 02:48:28,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: leader transfer failed - sending PoisonPill 02:48:28,639 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Leadership transfer expired 02:48:28,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Leadership transfer expired 02:48:28,640 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveShardReplica 02:48:28,640 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveShardReplica 02:48:28,640 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:28,640 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 02:48:28,641 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:48:28,641 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:48:28,643 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:28,643 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:28,648 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:48:28.650] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:28,671 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:48:28,679 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5278914102094129455] 02:48:28,679 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:48:28,682 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:48:28,682 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:48:28,688 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:28,688 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:28,693 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:28,695 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:28,695 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:48:28,695 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:48:28,695 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:48:28,697 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 02:48:28,705 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-698805418] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,709 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:48:28,719 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:48:28,719 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:28,720 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveAllShardReplicas 02:48:28,720 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:28,720 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:28,720 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 02:48:28,731 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 02:48:28,732 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@67d295ac 02:48:28,732 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 02:48:28,734 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:28,736 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:28,739 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:28,739 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:28,740 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:28,740 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:28,740 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas on shard member-1-shard-cars-config_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-80820785] 02:48:28,740 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveAllShardReplicas 02:48:28,741 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 02:48:28,741 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 02:48:28,741 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas on shard member-1-shard-people-config_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-2094215388] 02:48:28,742 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveAllShardReplicas, persistent : true 02:48:28,742 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-80820785] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,742 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#-604403957 created and ready for shard:member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:28,743 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#-604403957] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,743 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-900285494] 02:48:28,743 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 02:48:28,743 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-900285494] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,743 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveAllShardReplicas, persistent : true 02:48:28,743 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-2094215388] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,743 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-1318179871] 02:48:28,743 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#390486275 created and ready for shard:member-1-shard-people-config_testRemoveAllShardReplicas 02:48:28,744 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#390486275] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,744 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-1318179871] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,744 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 02:48:28,744 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 02:48:28,745 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:28,746 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:48:28,746 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:28,746 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 02:48:28,746 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveAllShardReplicas is initialized 02:48:28,747 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:28,747 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 02:48:28,747 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 02:48:28,747 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 02:48:28,747 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from null to Follower 02:48:28,749 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:28,749 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:28,749 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:48:28,749 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,750 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:28,750 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:28,751 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,751 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:28,751 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 02:48:28,751 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveAllShardReplicas is initialized 02:48:28,752 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 02:48:28,752 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#1877325057] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,753 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7fb4578d 02:48:28,754 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 02:48:28,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:28,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveAllShardReplicas is no longer the leader 02:48:28,757 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 02:48:28,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:28,757 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 02:48:28,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,759 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 02:48:28,759 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#1873830641] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,759 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#-460044209 created and ready for shard:member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,759 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/shard-member-1:snapshot-read#89646907] 02:48:28,759 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 02:48:28,760 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from null to Follower 02:48:28,760 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,760 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,760 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-1-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#1873830641] 02:48:28,760 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:28,760 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#-460044209] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,760 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:28,760 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:28,760 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-1-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#-1360912276] 02:48:28,760 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveAllShardReplicas, persistent : true 02:48:28,761 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#-2024212311 created and ready for shard:member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:28,761 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-1040770664] 02:48:28,761 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 02:48:28,762 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:28,763 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:28,763 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 02:48:28,763 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveAllShardReplicas is initialized 02:48:28,763 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:28,763 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 02:48:28,763 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 02:48:28,763 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 02:48:28,763 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from null to Follower 02:48:28,763 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:28,764 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:28,764 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 02:48:28,764 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveAllShardReplicas is initialized 02:48:28,764 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:28,764 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 02:48:28,764 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 02:48:28,764 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 02:48:28,765 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 02:48:28,767 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:48:28,776 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4553265249338731628] 02:48:28,776 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:48:28,778 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:28,778 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:48:28,782 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:28,782 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:28,791 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:28,792 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:28,794 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 02:48:28,806 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 02:48:28,807 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-88370756] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,813 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 02:48:28,813 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:28,813 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveAllShardReplicas 02:48:28,816 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 02:48:28,817 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@46aa564e 02:48:28,817 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 02:48:28,818 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:28,820 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:28,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:28,822 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:28,823 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveAllShardReplicas 02:48:28,823 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveAllShardReplicas, persistent : true 02:48:28,824 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 02:48:28,824 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 02:48:28,824 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveAllShardReplicas, persistent : true 02:48:28,825 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-715176559] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,825 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#1024867505 created and ready for shard:member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:28,825 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-1939080973] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,825 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#1024867505] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,825 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-624563032] 02:48:28,825 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 02:48:28,827 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 02:48:28,837 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-137201044] 02:48:28,838 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:28,838 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:28,838 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:28,838 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveAllShardReplicas is no longer the leader 02:48:28,838 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 02:48:28,838 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveAllShardReplicas is initialized 02:48:28,839 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 02:48:28,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:28,839 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:28,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:28,839 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 02:48:28,839 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveAllShardReplicas is initialized 02:48:28,841 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 02:48:28,842 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-624563032] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,842 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#1044080569] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,842 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-137201044] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,843 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2e3c74c5 02:48:28,843 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 02:48:28,845 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#-1554648249 created and ready for shard:member-2-shard-people-config_testRemoveAllShardReplicas 02:48:28,845 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 02:48:28,845 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 02:48:28,845 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:28,845 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,845 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#-1554648249] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,846 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:28,846 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveAllShardReplicas from null to Follower 02:48:28,846 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveAllShardReplicas, persistent : true 02:48:28,846 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 02:48:28,847 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 02:48:28,847 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-542938163] 02:48:28,847 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#-562905709 created and ready for shard:member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,847 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#-1881518727 created and ready for shard:member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:28,847 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/shard-member-2:snapshot-read#1813759178] 02:48:28,847 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas#202080152] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,847 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 02:48:28,847 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas#-1044437189] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,853 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:28,853 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 02:48:28,853 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 02:48:28,853 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 02:48:28,853 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveAllShardReplicas from null to Follower 02:48:28,854 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:28,854 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:28,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 02:48:28,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveAllShardReplicas is initialized 02:48:28,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:28,854 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:28,854 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:28,854 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 02:48:28,854 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 02:48:28,854 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 02:48:28,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:28,854 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveAllShardReplicas is initialized 02:48:28,854 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 02:48:28,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 02:48:28,855 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 02:48:28,855 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 02:48:28,855 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 02:48:28,855 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveAllShardReplicas from null to Follower 02:48:28,866 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:48:28,879 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:48:28,881 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:48:28,881 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:28,881 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:28,881 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:28,882 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 02:48:28,882 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 02:48:28,882 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:28,882 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 02:48:28,882 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:28,882 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,882 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,882 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:28,882 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,882 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:28,882 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:28,894 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [2487338415165635980] 02:48:28,894 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 02:48:28,896 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:28,896 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 02:48:28,897 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:28,897 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:28,911 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:28,913 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:28,915 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 02:48:28,919 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 02:48:28,921 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1481278635] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,938 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 02:48:28,938 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveAllShardReplicas 02:48:28,938 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveAllShardReplicas 02:48:28,942 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 02:48:28,943 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 02:48:28,945 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4bafca55 02:48:28,945 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 02:48:28,946 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:48:28,946 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:28,948 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:28,949 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:28,950 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:28,950 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 02:48:28,950 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:28,950 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveAllShardReplicas 02:48:28,953 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:28,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 02:48:28,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas on shard member-3-shard-cars-config_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#1171657808] 02:48:28,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveAllShardReplicas 02:48:28,953 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveAllShardReplicas, persistent : true 02:48:28,953 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#1171657808] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,953 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 02:48:28,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 02:48:28,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas on shard member-3-shard-people-config_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-2029736006] 02:48:28,954 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveAllShardReplicas, persistent : true 02:48:28,954 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-2029736006] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,954 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-2046250103] 02:48:28,954 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#206523349 created and ready for shard:member-3-shard-people-config_testRemoveAllShardReplicas 02:48:28,954 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#206523349] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,954 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-2046250103] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,955 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 02:48:28,955 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 02:48:28,955 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#1464719442 created and ready for shard:member-3-shard-cars-config_testRemoveAllShardReplicas 02:48:28,955 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-1496713631] 02:48:28,955 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#1464719442] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,955 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-1496713631] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,955 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 02:48:28,956 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#848202632] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,957 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 02:48:28,958 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:28,958 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:28,958 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:28,958 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testRemoveAllShardReplicas is no longer the leader 02:48:28,958 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 02:48:28,958 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 02:48:28,958 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 02:48:28,958 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveAllShardReplicas is initialized 02:48:28,958 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 02:48:28,958 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:28,958 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:28,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 02:48:28,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveAllShardReplicas is initialized 02:48:28,959 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 02:48:28,959 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,960 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveAllShardReplicas 02:48:28,960 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:28,960 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,960 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:28,961 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveAllShardReplicas from null to Follower 02:48:28,965 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:28,965 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 02:48:28,965 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 02:48:28,965 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:28,965 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 02:48:28,965 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveAllShardReplicas from null to Follower 02:48:28,968 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 02:48:28,968 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@58f56c59 02:48:28,969 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 02:48:28,969 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:28,969 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,969 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 02:48:28,970 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,970 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,970 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-3-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#1870759305] 02:48:28,970 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveAllShardReplicas 02:48:28,971 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 02:48:28,971 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 02:48:28,971 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveAllShardReplicas, persistent : true 02:48:28,972 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#-398391601 created and ready for shard:member-3-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,972 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/shard-member-3:snapshot-read#1999938321] 02:48:28,972 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#1870759305] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,972 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#440950970] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:28,972 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#664714800 created and ready for shard:member-3-shard-people-oper_testRemoveAllShardReplicas 02:48:28,973 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:28,973 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 02:48:28,973 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-3-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#440950970] 02:48:28,973 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/shard-member-3:snapshot-read#940969785] 02:48:28,973 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 02:48:28,973 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:28,973 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:28,973 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 02:48:28,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveAllShardReplicas is initialized 02:48:28,974 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:28,974 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 02:48:28,974 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 02:48:28,974 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:28,974 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 02:48:28,974 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 02:48:28,975 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:28,975 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:28,975 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 02:48:28,975 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveAllShardReplicas is initialized 02:48:28,975 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:28,975 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 02:48:28,975 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 02:48:28,976 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:28,976 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 02:48:28,976 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveAllShardReplicas from null to Follower 02:48:29,88 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:48:29,88 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:48:29,88 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 02:48:29,88 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 02:48:29,88 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testRemoveAllShardReplicas, member-3-shard-cars-oper_testRemoveAllShardReplicas] 02:48:29,88 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testRemoveAllShardReplicas, member-3-shard-cars-config_testRemoveAllShardReplicas] 02:48:29,88 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,88 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:29,88 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 02:48:29,88 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 02:48:29,89 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:29,89 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:29,89 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:29,89 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 02:48:29,89 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:29,89 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 02:48:29,89 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 02:48:29,89 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 02:48:29,89 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Follower to Candidate 02:48:29,90 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Follower to Candidate 02:48:29,117 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:48:29,117 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 02:48:29,117 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testRemoveAllShardReplicas, member-3-shard-people-oper_testRemoveAllShardReplicas] 02:48:29,117 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:29,117 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 02:48:29,118 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:29,118 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:29,118 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 02:48:29,118 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 02:48:29,118 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Follower to Candidate 02:48:29,137 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:48:29,137 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 02:48:29,138 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testRemoveAllShardReplicas, member-2-shard-people-config_testRemoveAllShardReplicas] 02:48:29,138 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 02:48:29,138 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 02:48:29,138 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:29,138 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:29,138 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveAllShardReplicas is no longer the leader 02:48:29,138 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 02:48:29,138 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Follower to Candidate 02:48:29,467 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 02:48:29,467 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 02:48:29,467 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 02:48:29,488 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 02:48:29,488 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,488 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 02:48:29,508 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 02:48:29,508 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 02:48:29,508 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:29,508 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:29,508 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 02:48:29,508 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 02:48:29,699 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 02:48:29,699 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:48:29,700 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:29,700 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,700 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,700 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:29,700 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:29,701 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 02:48:29,701 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:29,702 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:29,702 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,702 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,702 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:29,702 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:29,702 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,702 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:29,709 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:29,709 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,811 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:29,838 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 02:48:29,838 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:29,838 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 02:48:29,838 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:29,838 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testRemoveAllShardReplicas 02:48:29,840 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:29,840 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:29,840 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 02:48:29,840 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 02:48:29,840 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:29,840 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:29,840 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:29,840 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:29,840 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 02:48:29,840 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 02:48:29,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:29,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:29,841 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:29,841 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:29,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:29,841 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:29,842 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:48:29,843 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:48:29,843 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveAllShardReplicas, member-3-shard-people-oper_testRemoveAllShardReplicas] 02:48:29,843 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,843 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,843 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:48:29,843 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 02:48:29,843 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 02:48:29,843 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 02:48:29,843 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:29,843 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Candidate to Leader 02:48:29,844 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:29,844 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:29,844 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done false 02:48:29,844 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveAllShardReplicas status sync done false 02:48:29,844 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:29,844 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:29,845 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 02:48:29,845 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,845 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 02:48:29,845 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,848 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 02:48:29,848 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 02:48:29,848 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,848 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:29,848 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 02:48:29,848 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 02:48:29,848 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,848 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:29,848 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,848 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testRemoveAllShardReplicas 02:48:29,849 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:29,849 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:29,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:29,849 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 02:48:29,849 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 02:48:29,849 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:29,849 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:29,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 02:48:29,849 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:29,849 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:29,849 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 02:48:29,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:29,849 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 02:48:29,849 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:29,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 02:48:29,849 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:29,849 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:29,849 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:29,849 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:29,850 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:29,850 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,850 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:29,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:29,850 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:29,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 02:48:29,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:29,850 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:29,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 02:48:29,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:29,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:29,850 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:29,851 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:48:29,851 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:48:29,851 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:48:29,851 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:48:29,851 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveAllShardReplicas, member-3-shard-cars-config_testRemoveAllShardReplicas] 02:48:29,851 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,851 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testRemoveAllShardReplicas, member-3-shard-cars-oper_testRemoveAllShardReplicas] 02:48:29,851 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,851 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,851 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:48:29,851 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 02:48:29,851 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,852 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: became leader with frontend state for [] 02:48:29,852 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:48:29,852 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 02:48:29,852 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 02:48:29,852 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:29,852 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 02:48:29,852 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:29,852 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Candidate to Leader 02:48:29,852 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 02:48:29,852 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Candidate to Leader 02:48:29,852 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 02:48:29,853 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:29,853 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,853 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:29,853 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:29,853 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:29,853 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 02:48:29,853 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done false 02:48:29,853 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done false 02:48:29,853 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,853 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:29,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:29,853 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 02:48:29,853 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveAllShardReplicas status sync done false 02:48:29,853 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveAllShardReplicas status sync done false 02:48:29,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 02:48:29,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 02:48:29,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 02:48:29,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,854 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 02:48:29,854 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,877 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 02:48:29,877 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 02:48:29,877 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 02:48:29,877 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testRemoveAllShardReplicas 02:48:29,877 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testRemoveAllShardReplicas 02:48:29,879 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:29,879 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 02:48:29,879 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:29,879 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:29,879 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:29,879 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 02:48:29,879 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 02:48:29,879 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:29,879 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:29,879 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 02:48:29,879 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:29,879 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:29,879 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 02:48:29,879 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:29,880 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 02:48:29,880 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:29,881 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:48:29,881 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:48:29,881 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveAllShardReplicas, member-2-shard-people-config_testRemoveAllShardReplicas] 02:48:29,881 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,881 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,882 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:48:29,882 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 02:48:29,882 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: became leader with frontend state for [] 02:48:29,882 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 02:48:29,882 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:29,882 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Candidate to Leader 02:48:29,882 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 02:48:29,882 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} is now ready 02:48:29,882 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-698805418] 02:48:29,882 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:29,882 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-80820785] 02:48:29,883 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-80820785] 02:48:29,883 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-88370756] 02:48:29,883 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:29,884 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-715176559] 02:48:29,884 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-715176559] 02:48:29,884 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 02:48:29,884 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 02:48:29,884 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveAllShardReplicas status sync done false 02:48:29,884 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:29,884 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:29,884 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 02:48:29,884 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done false 02:48:29,885 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 02:48:29,885 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1481278635] 02:48:29,885 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 02:48:29,885 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,885 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 02:48:29,885 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:29,885 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:29,886 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#1171657808] 02:48:29,886 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#1171657808] 02:48:29,897 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@754c3cfd] 02:48:29,898 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 02:48:29,899 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Shard created : member-1-shard-pets-config_testRemoveAllShardReplicas, persistent : true 02:48:29,901 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-1529184484] 02:48:29,901 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Starting recovery for member-1-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 02:48:29,901 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas-notifier#1363716411 created and ready for shard:member-1-shard-pets-config_testRemoveAllShardReplicas 02:48:29,903 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:29,903 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:29,903 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 02:48:29,903 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testRemoveAllShardReplicas is initialized 02:48:29,903 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:29,903 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 02:48:29,903 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 02:48:29,903 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 02:48:29,903 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from null to Follower 02:48:29,910 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@2428fbc0] 02:48:29,911 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-1-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 02:48:29,911 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 02:48:29,912 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-2-shard-pets-config_testRemoveAllShardReplicas, persistent : true 02:48:29,913 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-251611838] 02:48:29,913 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas-notifier#1096392702 created and ready for shard:member-2-shard-pets-config_testRemoveAllShardReplicas 02:48:29,913 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-2-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 02:48:29,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:29,914 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:29,914 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 02:48:29,914 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testRemoveAllShardReplicas is initialized 02:48:29,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:29,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 02:48:29,914 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 02:48:29,914 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 02:48:29,914 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testRemoveAllShardReplicas from null to Follower 02:48:29,922 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@60da8a95] 02:48:29,923 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3 doCreateShard: shardId: member-3-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-1-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 02:48:29,923 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-698805418] 02:48:29,924 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-3-shard-pets-config_testRemoveAllShardReplicas, persistent : true 02:48:29,924 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 02:48:29,924 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#2146752585] 02:48:29,924 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#2146752585] 02:48:29,924 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#1254132040] 02:48:29,924 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas-notifier#-731454721 created and ready for shard:member-3-shard-pets-config_testRemoveAllShardReplicas 02:48:29,924 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-3-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 02:48:29,925 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-88370756] 02:48:29,925 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 02:48:29,925 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1523201115] 02:48:29,925 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1523201115] 02:48:29,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:29,925 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1481278635] 02:48:29,925 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:29,925 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [pets] 02:48:29,926 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized 02:48:29,926 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 02:48:29,926 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:29,926 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 02:48:29,926 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 02:48:29,926 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-1762123143] 02:48:29,926 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 02:48:29,926 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-1762123143] 02:48:29,926 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-pets-config_testRemoveAllShardReplicas from null to Follower 02:48:29,927 PM [main] [INFO] ClusterAdminRpcService - Removing replicas for all shards 02:48:29,927 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testRemoveAllShardReplicas 02:48:29,927 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 02:48:29,927 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testRemoveAllShardReplicas 02:48:29,927 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 02:48:29,928 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 02:48:29,928 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=pets, memberName=MemberName{name=member-3}] 02:48:29,928 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:29,928 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, primaryVersion=13] 02:48:29,928 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 02:48:29,928 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 02:48:29,928 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, primaryVersion=13] 02:48:29,928 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 02:48:29,928 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:29,928 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, primaryVersion=13] 02:48:29,929 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 02:48:29,929 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, primaryVersion=13] 02:48:29,929 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas for shard member-3-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,929 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas for shard member-3-shard-people-oper_testRemoveAllShardReplicas 02:48:29,932 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-oper_testRemoveAllShardReplicas'}, state: Idle 02:48:29,932 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-people-oper_testRemoveAllShardReplicas'}, state: Idle 02:48:29,932 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]] 02:48:29,932 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]] 02:48:29,932 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,932 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,932 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,932 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,932 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=7cc1eef8-3b0c-4351-a31c-b8413c0344f4}, logIndex: 0, isSendImmediate: true 02:48:29,932 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=eb06d1e8-1935-4a22-8793-88652bada3d6}, logIndex: 0, isSendImmediate: true 02:48:29,932 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:29,932 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:29,932 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 02:48:29,932 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:29,929 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 02:48:29,932 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,932 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-people-oper_testRemoveAllShardReplicas'} 02:48:29,932 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 02:48:29,932 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-oper_testRemoveAllShardReplicas'} 02:48:29,933 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,933 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,934 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas for shard member-3-shard-cars-config_testRemoveAllShardReplicas 02:48:29,934 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas for shard member-3-shard-people-config_testRemoveAllShardReplicas 02:48:29,934 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 02:48:29,934 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 02:48:29,934 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:29,934 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 02:48:29,934 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 02:48:29,934 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,934 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 02:48:29,934 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:29,934 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 02:48:29,934 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, votingState=VOTING]] 02:48:29,934 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Log size is now 1 02:48:29,934 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,934 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:29,934 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-oper_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, votingState=VOTING]] 02:48:29,934 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Log size is now 1 02:48:29,934 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:29,935 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-oper_testRemoveAllShardReplicas} for shard people 02:48:29,935 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard people 02:48:29,935 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testRemoveAllShardReplicas} for shard cars 02:48:29,935 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,935 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard cars 02:48:29,935 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 02:48:29,935 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 1 02:48:29,935 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 02:48:29,935 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done true 02:48:29,935 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 2 02:48:29,935 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,935 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done true 02:48:29,935 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-config_testRemoveAllShardReplicas'}, state: Idle 02:48:29,935 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]] 02:48:29,935 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,936 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,936 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-people-config_testRemoveAllShardReplicas'}, state: Idle 02:48:29,936 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]] 02:48:29,936 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=edea8651-48dc-442d-a38a-e808505c4370}, logIndex: 0, isSendImmediate: true 02:48:29,936 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,936 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,936 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:29,936 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:29,936 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=af350360-557b-4e44-90ed-b19ab3c183ab}, logIndex: 0, isSendImmediate: true 02:48:29,936 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 02:48:29,936 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:29,936 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-config_testRemoveAllShardReplicas'} 02:48:29,936 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testRemoveAllShardReplicas 02:48:29,936 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-people-config_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 02:48:29,936 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-people-config_testRemoveAllShardReplicas'} 02:48:29,936 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,936 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,937 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 02:48:29,937 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 02:48:29,937 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:48:29,937 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:48:29,937 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Setting last applied to 0 02:48:29,937 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Setting last applied to 0 02:48:29,937 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 02:48:29,937 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 02:48:29,937 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 02:48:29,937 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 02:48:29,937 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:29,937 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:29,937 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:29,937 PM [cluster-test-shard-dispatcher-24] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:29,937 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 02:48:29,937 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 02:48:29,938 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 02:48:29,938 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 02:48:29,938 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 02:48:29,938 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 02:48:29,938 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:29,938 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:29,938 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 02:48:29,938 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 02:48:29,938 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,938 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,938 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Commit index set to 0 02:48:29,938 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, votingState=VOTING]] 02:48:29,938 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Log size is now 1 02:48:29,938 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, votingState=VOTING]] 02:48:29,938 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:29,938 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Log size is now 1 02:48:29,938 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:29,938 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:29,938 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Setting last applied to 0 02:48:29,938 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 02:48:29,938 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 02:48:29,938 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Commit index set to 0 02:48:29,938 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:29,938 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Setting last applied to 0 02:48:29,938 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 02:48:29,938 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 02:48:29,938 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 02:48:29,938 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 02:48:29,939 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,939 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done true 02:48:29,937 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveAllShardReplicas} for shard cars 02:48:29,939 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 02:48:29,939 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:29,939 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-config_testRemoveAllShardReplicas} for shard people 02:48:29,939 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done true 02:48:29,939 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 02:48:29,939 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard people 02:48:29,939 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 3 02:48:29,939 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 02:48:29,938 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#1870759305] with 27000 ms timeout 02:48:29,939 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 4 02:48:29,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard cars has been removed 02:48:29,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [people] 02:48:29,940 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 02:48:29,940 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#440950970] with 27000 ms timeout 02:48:29,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 02:48:29,940 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:48:29,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:48:29,940 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard people has been removed 02:48:29,940 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 02:48:29,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 02:48:29,940 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [] 02:48:29,940 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 02:48:29,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 02:48:29,940 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 02:48:29,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 02:48:29,940 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onShutDown 02:48:29,940 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:29,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:29,940 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:29,940 PM [cluster-test-shard-dispatcher-24] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:29,940 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 02:48:29,940 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 02:48:29,940 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveAllShardReplicas 02:48:29,940 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:29,941 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 02:48:29,941 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#440950970] 02:48:29,941 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:29,941 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:29,941 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 02:48:29,941 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:29,941 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 02:48:29,941 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 02:48:29,941 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-2029736006] with 27000 ms timeout 02:48:29,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard people has been removed 02:48:29,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets, cars] 02:48:29,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 02:48:29,942 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onShutDown 02:48:29,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:29,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 02:48:29,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 02:48:29,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#1171657808] with 27000 ms timeout 02:48:29,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 02:48:29,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 02:48:29,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 02:48:29,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:29,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 02:48:29,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 02:48:29,942 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveAllShardReplicas 02:48:29,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 02:48:29,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 02:48:29,942 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:29,942 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#1870759305] 02:48:29,942 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-2029736006] 02:48:29,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 02:48:29,942 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onShutDown 02:48:29,942 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets] 02:48:29,943 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:29,943 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveAllShardReplicas 02:48:29,943 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:29,943 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:29,943 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 02:48:29,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#1171657808] 02:48:29,943 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 02:48:30,318 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:48:30,318 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 02:48:30,318 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas] 02:48:30,318 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-pets-config_testRemoveAllShardReplicas 02:48:30,318 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 02:48:30,318 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-pets-config_testRemoveAllShardReplicas 02:48:30,319 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-pets-config_testRemoveAllShardReplicas 02:48:30,319 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:30,319 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 02:48:30,319 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 02:48:30,319 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 02:48:30,320 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Follower to Candidate 02:48:30,320 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:30,320 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 02:48:30,320 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:30,320 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:30,320 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=null 02:48:30,321 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 1, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:30,321 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-pets-config_testRemoveAllShardReplicas 02:48:30,321 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=1, voteGranted=true] 02:48:30,322 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=1, voteGranted=true], my term is 1 02:48:30,322 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:30,322 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=1, voteGranted=true], current voteCount: 1 02:48:30,323 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 02:48:30,323 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:30,323 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:30,323 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=null 02:48:30,323 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas] 02:48:30,323 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 1, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:30,323 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-pets-config_testRemoveAllShardReplicas 02:48:30,323 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=1, voteGranted=true] 02:48:30,323 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:30,323 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:30,323 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:48:30,323 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 02:48:30,323 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: became leader with frontend state for [] 02:48:30,324 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 02:48:30,324 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:30,324 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Candidate to Leader 02:48:30,324 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 02:48:30,324 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-pets-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas 02:48:30,324 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:30,325 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 02:48:30,325 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-pets-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas 02:48:30,325 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done false 02:48:30,325 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 02:48:30,325 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 02:48:30,325 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for pets: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, primaryVersion=13] 02:48:30,325 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 02:48:30,325 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-pets-config_testRemoveAllShardReplicas status sync done false 02:48:30,326 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas for shard member-3-shard-pets-config_testRemoveAllShardReplicas 02:48:30,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 02:48:30,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:30,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 02:48:30,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:30,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-pets-config_testRemoveAllShardReplicas'}, state: Idle 02:48:30,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]] 02:48:30,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:30,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:30,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=c903826d-260d-499e-93e4-05bb841b42ec}, logIndex: 0, isSendImmediate: true 02:48:30,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:30,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-pets-config_testRemoveAllShardReplicas 02:48:30,328 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-pets-config_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 02:48:30,328 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-pets-config_testRemoveAllShardReplicas'} 02:48:30,328 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:30,328 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 02:48:30,329 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 02:48:30,329 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:30,329 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 02:48:30,329 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:30,329 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-pets-config_testRemoveAllShardReplicas} for shard pets 02:48:30,329 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard pets 02:48:30,329 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClusterAdminRpcService - onSuccess for shard pets, type Config 02:48:30,329 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 5 02:48:30,329 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-pets-config_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, votingState=VOTING]] 02:48:30,329 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Log size is now 1 02:48:30,329 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-pets-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:30,329 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 02:48:30,330 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done true 02:48:30,330 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-698805418] 02:48:30,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:30,331 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-80820785] 02:48:30,331 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-80820785] 02:48:30,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 02:48:30,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:48:30,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 02:48:30,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 02:48:30,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 02:48:30,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:30,331 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:30,331 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-698805418] 02:48:30,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 02:48:30,331 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 02:48:30,331 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-2094215388] 02:48:30,331 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-2094215388] 02:48:30,332 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-698805418] 02:48:30,332 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 02:48:30,332 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 02:48:30,332 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:30,332 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 02:48:30,332 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 02:48:30,332 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 02:48:30,332 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 02:48:30,332 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#2146752585] 02:48:30,332 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#2146752585] 02:48:30,333 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-1762123143] with 27000 ms timeout 02:48:30,333 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard pets has been removed 02:48:30,333 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [] 02:48:30,333 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onShutDown 02:48:30,333 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:30,333 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-88370756] 02:48:30,333 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-3-shard-pets-config_testRemoveAllShardReplicas 02:48:30,333 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:30,333 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:30,333 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-715176559] 02:48:30,333 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-715176559] 02:48:30,333 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-1762123143] 02:48:30,334 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-88370756] 02:48:30,334 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 02:48:30,334 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-1939080973] 02:48:30,334 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 02:48:30,334 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-1939080973] 02:48:30,334 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-88370756] 02:48:30,334 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 02:48:30,334 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1523201115] 02:48:30,334 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1523201115] 02:48:30,334 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1481278635] 02:48:30,334 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:30,334 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [] 02:48:30,334 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1481278635] 02:48:30,334 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 02:48:30,334 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard people not found - shards present: [] 02:48:30,334 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1481278635] 02:48:30,334 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 02:48:30,334 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard pets not found - shards present: [] 02:48:30,334 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 02:48:30,335 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 0 shards 02:48:30,335 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 02:48:30,335 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 02:48:30,335 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 02:48:30,335 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 0 shards 02:48:30,335 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 02:48:30,335 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 02:48:30,336 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:30,336 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:30,342 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:30,343 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:30,343 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:30,343 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:30,345 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:48:30,351 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 02:48:30,352 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testRemoveAllShardReplicas 02:48:30,352 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:30,352 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onShutDown 02:48:30,352 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveAllShardReplicas 02:48:30,352 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 02:48:30,352 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onShutDown 02:48:30,352 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onShutDown 02:48:30,352 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-2-shard-pets-config_testRemoveAllShardReplicas 02:48:30,352 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:30,352 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveAllShardReplicas 02:48:30,352 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveAllShardReplicas 02:48:30,352 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:30,352 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:30,353 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 02:48:30,353 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 02:48:30,353 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 02:48:30,353 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:30,353 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:30,353 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 02:48:30,354 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 02:48:30,354 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onShutDown 02:48:30,354 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 02:48:30,354 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:30,354 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveAllShardReplicas 02:48:30,354 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:30,354 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 02:48:30,354 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 02:48:30,355 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:30,355 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:30,718 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 02:48:31,358 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [2487338415165635980] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:48:31,368 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-4553265249338731628] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:48:31,377 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:31,378 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:31,379 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:48:31.382] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:31,384 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 02:48:31,384 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testRemoveAllShardReplicas 02:48:31,384 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:31,384 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveAllShardReplicas 02:48:31,384 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 02:48:31,384 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onShutDown 02:48:31,384 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Initiating leader transfer 02:48:31,384 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onShutDown 02:48:31,384 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Initiating leader transfer 02:48:31,384 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onShutDown 02:48:31,384 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Initiating leader transfer 02:48:31,385 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@a172a7c 02:48:31,385 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2603eceb 02:48:31,385 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2603eceb 02:48:31,385 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-pets-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@a172a7c 02:48:31,385 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 02:48:31,385 PM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 02:48:31,385 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 02:48:31,385 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 02:48:31,385 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@61a7fb2d 02:48:31,385 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@61a7fb2d 02:48:31,385 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 02:48:31,385 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 02:48:31,385 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:31,385 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:31,385 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:31,385 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 02:48:31,385 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 02:48:31,385 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 02:48:31,768 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer aborted 02:48:31,768 PM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: Failed to transfer leadership in 383.5 ms 02:48:31,768 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In unpauseLeader 02:48:31,769 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 02:48:31,769 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 02:48:31,769 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveAllShardReplicas 02:48:31,769 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:31,797 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer aborted 02:48:31,797 PM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: Failed to transfer leadership in 412.8 ms 02:48:31,797 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In unpauseLeader 02:48:31,798 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 02:48:31,798 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 02:48:31,798 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveAllShardReplicas 02:48:31,798 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:31,918 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer aborted 02:48:31,918 PM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: Failed to transfer leadership in 533.8 ms 02:48:31,918 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In unpauseLeader 02:48:31,918 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 02:48:31,918 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 02:48:31,919 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-pets-config_testRemoveAllShardReplicas 02:48:31,919 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:31,920 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:48:31,920 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:48:31,920 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 02:48:31,921 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:31,921 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:31,921 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 02:48:31,921 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onShutDown 02:48:31,921 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Initiating leader transfer 02:48:31,921 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 02:48:31,921 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Initiating leader transfer 02:48:31,921 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 02:48:31,921 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@45d96811 02:48:31,921 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@45d96811 02:48:31,921 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 02:48:31,921 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6b72aeb 02:48:31,921 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 02:48:31,921 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6b72aeb 02:48:31,921 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 02:48:31,921 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 02:48:31,921 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 02:48:31,921 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:31,922 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:32,398 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer aborted 02:48:32,398 PM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 476.9 ms 02:48:32,398 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In unpauseLeader 02:48:32,398 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 02:48:32,398 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Leadership transfer expired 02:48:32,399 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveAllShardReplicas 02:48:32,399 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:32,407 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer aborted 02:48:32,407 PM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 485.9 ms 02:48:32,407 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In unpauseLeader 02:48:32,407 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 02:48:32,407 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Leadership transfer expired 02:48:32,408 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 02:48:32,408 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 02:48:32,409 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:48:32,409 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:48:32,410 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:32,410 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:32,413 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:48:32.416] [cluster-test-akka.actor.internal-dispatcher-5] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:32,438 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 02:48:32,454 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-576416124525691061] 02:48:32,455 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:48:32,459 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:48:32,459 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:48:32,461 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:32,461 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:32,474 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:48:32,475 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:32,475 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:48:32,475 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:48:32,476 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:32,478 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplicaFailures 02:48:32,484 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-873351030] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,484 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:48:32,485 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:48:32,485 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplicaFailures 02:48:32,485 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:32,485 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 02:48:32,490 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 02:48:32,491 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@12655131 02:48:32,491 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} started 02:48:32,492 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:32,493 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:32,494 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplicaFailures 02:48:32,495 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#-1821678875] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,496 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:32,496 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplicaFailures 02:48:32,496 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 02:48:32,496 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures to akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 02:48:32,496 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplicaFailures on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures on shard member-1-shard-cars-config_testAddShardReplicaFailures whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#465613725] 02:48:32,496 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddShardReplicaFailures, persistent : true 02:48:32,497 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#465613725] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/shard-member-1:snapshot-read#1198416570] 02:48:32,497 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#1755895863 created and ready for shard:member-1-shard-cars-config_testAddShardReplicaFailures 02:48:32,497 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#1755895863] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,497 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddShardReplicaFailures with journal batch size 1 02:48:32,497 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/shard-member-1:snapshot-read#1198416570] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,498 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:48:32,501 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:32,501 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:48:32,501 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:32,501 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 02:48:32,501 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplicaFailures is initialized 02:48:32,501 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplicaFailures 02:48:32,502 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:32,502 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 02:48:32,502 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 02:48:32,502 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplicaFailures is no longer the leader 02:48:32,502 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from null to Follower 02:48:32,502 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Follower): Received TimeoutNow - switching to Candidate 02:48:32,502 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Election: Candidate has following voting peers: [] 02:48:32,502 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Set currentTerm=1, votedFor=member-1-shard-cars-config_testAddShardReplicaFailures 02:48:32,502 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Starting new election term 1 02:48:32,502 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:32,502 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 02:48:32,502 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplicaFailures is no longer the leader 02:48:32,502 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Follower to Candidate 02:48:32,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Received ElectionTimeout 02:48:32,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Leader): Election: Leader has following peers: [] 02:48:32,503 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:48:32,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 02:48:32,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: became leader with frontend state for [] 02:48:32,503 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Candidate to Leader 02:48:32,503 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures 02:48:32,503 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplicaFailures, leaderId=member-1-shard-cars-config_testAddShardReplicaFailures, leaderPayloadVersion=13] 02:48:32,503 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplicaFailures from Candidate to Leader 02:48:32,503 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplicaFailures is ready 02:48:32,504 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 02:48:32,505 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3398ac7f 02:48:32,505 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} started 02:48:32,506 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:32,506 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplicaFailures 02:48:32,507 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 02:48:32,507 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures to akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 02:48:32,507 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplicaFailures on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures on shard member-1-shard-cars-oper_testAddShardReplicaFailures whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-835842968] 02:48:32,507 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-835842968] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,507 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddShardReplicaFailures, persistent : true 02:48:32,508 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#-1049729437 created and ready for shard:member-1-shard-cars-oper_testAddShardReplicaFailures 02:48:32,508 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#-1049729437] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,508 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/shard-member-1:snapshot-read#-1319693237] 02:48:32,508 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/shard-member-1:snapshot-read#-1319693237] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,509 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard people 02:48:32,509 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: onAddShardReplica: AddShardReplica[ShardName=people] 02:48:32,509 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: No module configuration exists for shard people 02:48:32,509 PM [cluster-test-akka.actor.default-dispatcher-4] [ERROR] ClusterAdminRpcService - Failed to add replica for shard people java.lang.IllegalArgumentException: No module configuration exists for shard people at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onAddShardReplica(ShardManager.java:1188) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:241) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 02:48:32,509 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddShardReplicaFailures with journal batch size 1 02:48:32,510 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 02:48:32,510 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplicaFailures 02:48:32,510 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 02:48:32,510 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onShutDown 02:48:32,510 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@a6feee0 02:48:32,510 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplicaFailures: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@a6feee0 02:48:32,510 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:32,510 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:32,511 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 02:48:32,511 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplicaFailures is initialized 02:48:32,511 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 02:48:32,511 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplicaFailures is no longer the leader 02:48:32,511 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from null to Follower 02:48:32,511 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower): Received TimeoutNow - switching to Candidate 02:48:32,511 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Election: Candidate has following voting peers: [] 02:48:32,511 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testAddShardReplicaFailures 02:48:32,511 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Starting new election term 1 02:48:32,511 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:32,511 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 02:48:32,511 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplicaFailures is no longer the leader 02:48:32,511 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Follower to Candidate 02:48:32,511 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddShardReplicaFailures 02:48:32,511 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Received ElectionTimeout 02:48:32,511 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Leader): Election: Leader has following peers: [] 02:48:32,512 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:48:32,512 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 02:48:32,512 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: became leader with frontend state for [] 02:48:32,512 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Candidate to Leader 02:48:32,512 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , registered listener akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures 02:48:32,512 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddShardReplicaFailures, leaderId=member-1-shard-cars-oper_testAddShardReplicaFailures, leaderPayloadVersion=13] 02:48:32,512 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplicaFailures from Candidate to Leader 02:48:32,512 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplicaFailures is ready 02:48:32,512 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Aborting 0 pending queued transactions 02:48:32,512 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:48:32,512 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:48:32,512 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 02:48:32,513 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-873351030] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-873351030] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,513 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplicaFailures 02:48:32,513 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 02:48:32,513 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onShutDown 02:48:32,513 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@3ade3af1 02:48:32,513 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplicaFailures: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@3ade3af1 02:48:32,513 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-873351030] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-873351030] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,514 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddShardReplicaFailures 02:48:32,514 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Aborting 0 pending queued transactions 02:48:32,514 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:48:32,514 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:48:32,515 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:32,515 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:32,516 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:48:32,533 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:48:32,543 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-4096511887538692089] 02:48:32,543 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:48:32,546 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:48:32,546 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:48:32,561 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:32,561 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:32,588 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:48:32,589 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:48:32,589 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:48:32,589 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:32,592 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:32,596 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 02:48:32,601 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1994489659] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,602 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:48:32,606 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:48:32,606 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplica 02:48:32,607 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:32,607 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 02:48:32,609 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 02:48:32,624 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@64ac3cb6 02:48:32,624 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} started 02:48:32,627 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:32,627 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplica 02:48:32,629 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 02:48:32,629 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 02:48:32,629 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica on shard member-1-shard-cars-config_testAddShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187] 02:48:32,630 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddShardReplica, persistent : true 02:48:32,631 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,638 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/shard-member-1:snapshot-read#27176218] 02:48:32,638 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#-1528197894 created and ready for shard:member-1-shard-cars-config_testAddShardReplica 02:48:32,638 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#-1528197894] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,640 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/shard-member-1:snapshot-read#27176218] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,640 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddShardReplica with journal batch size 1 02:48:32,642 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:32,642 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:32,642 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 02:48:32,642 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplica is initialized 02:48:32,642 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:32,642 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplica is no longer the leader 02:48:32,642 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from null to Follower 02:48:32,642 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Follower): Received TimeoutNow - switching to Candidate 02:48:32,642 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Election: Candidate has following voting peers: [] 02:48:32,642 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-config_testAddShardReplica 02:48:32,642 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Starting new election term 1 02:48:32,642 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:32,642 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:32,643 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplica is no longer the leader 02:48:32,643 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Follower to Candidate 02:48:32,643 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Received ElectionTimeout 02:48:32,643 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Election: Leader has following peers: [] 02:48:32,644 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:48:32,644 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:32,644 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: became leader with frontend state for [] 02:48:32,644 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Candidate to Leader 02:48:32,644 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplica 02:48:32,644 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 02:48:32,644 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplica from Candidate to Leader 02:48:32,644 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 02:48:32,677 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1994489659] 02:48:32,677 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:32,677 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187] 02:48:32,678 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:32,682 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:32,686 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 02:48:32,688 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-787511874] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,688 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:48:32,689 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:48:32,689 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplica 02:48:32,690 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:32,690 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 02:48:32,693 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 02:48:32,694 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3a596886 02:48:32,694 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} started 02:48:32,695 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:32,695 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplica 02:48:32,695 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 02:48:32,695 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 02:48:32,695 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica on shard member-1-shard-cars-oper_testAddShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999] 02:48:32,696 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddShardReplica, persistent : true 02:48:32,696 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,696 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/shard-member-1:snapshot-read#-4697512] 02:48:32,696 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#571268858 created and ready for shard:member-1-shard-cars-oper_testAddShardReplica 02:48:32,696 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#571268858] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,696 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/shard-member-1:snapshot-read#-4697512] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,697 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddShardReplica with journal batch size 1 02:48:32,697 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:32,697 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:32,697 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 02:48:32,697 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplica is initialized 02:48:32,697 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:32,697 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplica is no longer the leader 02:48:32,697 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from null to Follower 02:48:32,697 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Follower): Received TimeoutNow - switching to Candidate 02:48:32,698 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Election: Candidate has following voting peers: [] 02:48:32,698 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testAddShardReplica 02:48:32,698 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Starting new election term 1 02:48:32,698 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:32,698 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:32,698 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplica is no longer the leader 02:48:32,698 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Follower to Candidate 02:48:32,698 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Received ElectionTimeout 02:48:32,698 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Election: Leader has following peers: [] 02:48:32,698 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:48:32,698 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:32,698 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: became leader with frontend state for [] 02:48:32,698 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Candidate to Leader 02:48:32,698 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testAddShardReplica 02:48:32,699 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 02:48:32,699 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Candidate to Leader 02:48:32,699 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 02:48:32,745 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-787511874] 02:48:32,745 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 02:48:32,745 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999] 02:48:32,756 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:48:32,769 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4656462791534312841] 02:48:32,769 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:48:32,773 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:32,773 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:48:32,773 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:32,774 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:32,781 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:32,787 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:32,789 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 02:48:32,794 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 02:48:32,797 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1105992632] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,801 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 02:48:32,804 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 02:48:32,806 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@77e2e8bd 02:48:32,807 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} started 02:48:32,808 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:32,808 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:32,809 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:32,813 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 02:48:32,815 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 02:48:32,816 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 02:48:32,816 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#497785724] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:32,818 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:48:32,818 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 02:48:32,819 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7d1a83cd 02:48:32,820 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} started 02:48:32,820 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:32,822 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:48:32,825 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:32,825 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 02:48:32,825 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:32,825 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 02:48:33,572 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:48:33,572 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:33,572 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 02:48:33,572 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:33,572 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 02:48:33,572 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 02:48:33,572 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 02:48:33,572 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 02:48:33,573 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 02:48:33,622 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 02:48:33,622 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 02:48:33,623 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:33,623 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica] 02:48:33,626 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 02:48:33,626 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187, primaryVersion=13] 02:48:33,629 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-2-shard-cars-config_testAddShardReplica, persistent : true 02:48:33,630 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187 for shard member-2-shard-cars-config_testAddShardReplica 02:48:33,630 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1728454553] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:33,630 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/shard-member-2:snapshot-read#-1090768941] 02:48:33,631 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#-1242903458 created and ready for shard:member-2-shard-cars-config_testAddShardReplica 02:48:33,631 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testAddShardReplica with journal batch size 1 02:48:33,631 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#-1242903458] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:33,631 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/shard-member-2:snapshot-read#-1090768941] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:33,632 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:33,632 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:33,632 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:33,632 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddShardReplica is no longer the leader 02:48:33,633 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , received role change from null to Follower 02:48:33,633 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 02:48:33,633 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddShardReplica is initialized 02:48:33,633 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true], state: Idle 02:48:33,633 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplica 02:48:33,633 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true] 02:48:33,633 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:48:33,633 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddShardReplica from null to Follower 02:48:33,633 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddShardReplica 02:48:33,633 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: lastSequenceNumber prior to capture: 1 02:48:33,633 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:48:33,633 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating capture snapshot for new server member-2-shard-cars-config_testAddShardReplica 02:48:33,634 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: CaptureSnapshotReply received by actor 02:48:33,634 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2d96cbde, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddShardReplica, ServerConfigPayload=null] 02:48:33,634 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 02:48:33,635 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendInstallSnapshot 02:48:33,635 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testAddShardReplica: 229 02:48:33,635 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddShardReplica, member-2-shard-cars-config_testAddShardReplica), Chunk: 1/1 02:48:33,635 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:48:33,635 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 1 02:48:33,635 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: SnapshotComplete received 02:48:33,635 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a snapshot response 02:48:33,635 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a journal response 02:48:33,637 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 02:48:33,637 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 02:48:33,637 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 02:48:33,637 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 02:48:33,637 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-config_testAddShardReplica 02:48:33,637 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-config_testAddShardReplica to state 02:48:33,638 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 02:48:33,638 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=-1] 02:48:33,638 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 02:48:33,638 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2518efce, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 02:48:33,638 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 02:48:33,638 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 02:48:33,639 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:48:33,639 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 1 02:48:33,639 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to -1 02:48:33,639 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 02:48:33,639 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Applying snapshot 02:48:33,639 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 02:48:33,639 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#1202285110] with name member-2-shard-cars-config_testAddShardReplica-DTCL-publisher 02:48:33,639 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#1202285110] 02:48:33,639 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: state snapshot applied in 583.3 μs 02:48:33,639 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Done applying snapshot 02:48:33,639 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 02:48:33,640 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 02:48:33,640 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: SnapshotComplete received 02:48:33,640 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: handled a snapshot response 02:48:33,640 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#1202285110] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:33,640 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 184.8 μs 02:48:33,640 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: handled a journal response 02:48:33,641 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 02:48:33,641 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 02:48:33,641 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 02:48:33,641 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testAddShardReplica] 02:48:33,641 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]] 02:48:33,641 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:33,641 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:33,641 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=eafa9411-b9bc-4322-a0c5-66a9d6a97eef}, logIndex: 0, isSendImmediate: true 02:48:33,641 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:33,641 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testAddShardReplica 02:48:33,641 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 02:48:33,642 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true] 02:48:33,642 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:33,642 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 02:48:33,643 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:33,643 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:33,643 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 02:48:33,643 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:33,643 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187 02:48:33,643 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 02:48:33,643 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddShardReplica 02:48:33,643 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 02:48:33,643 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 02:48:33,644 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1105992632] 02:48:33,644 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:33,644 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:33,644 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1728454553] 02:48:33,644 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1728454553] 02:48:33,644 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 02:48:33,644 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 02:48:33,644 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 1 02:48:33,644 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:33,644 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 02:48:33,644 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 02:48:33,645 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 02:48:33,645 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:33,645 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 0, nextIndex: 1 02:48:33,645 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#497785724] 02:48:33,645 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:48:33,645 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 0 02:48:33,645 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 02:48:33,645 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 02:48:33,645 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:33,645 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 02:48:33,645 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddShardReplica 02:48:33,646 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 02:48:33,646 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddShardReplica is no longer the leader 02:48:33,646 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 02:48:33,646 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 02:48:33,646 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 02:48:33,646 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:33,646 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=0 02:48:33,646 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 02:48:33,646 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 02:48:33,647 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:33,647 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica] 02:48:33,647 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 0 02:48:33,647 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:33,647 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 0 02:48:33,647 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 02:48:33,647 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 02:48:33,647 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=0 02:48:33,647 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 02:48:33,648 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 02:48:33,648 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999, primaryVersion=13] 02:48:33,650 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999 for shard member-2-shard-cars-oper_testAddShardReplica 02:48:33,650 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-2-shard-cars-oper_testAddShardReplica, persistent : true 02:48:33,650 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:33,651 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/shard-member-2:snapshot-read#-1332283892] 02:48:33,651 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#-483953545 created and ready for shard:member-2-shard-cars-oper_testAddShardReplica 02:48:33,651 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#-483953545] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:33,651 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true], state: Idle 02:48:33,651 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/shard-member-2:snapshot-read#-1332283892] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:33,651 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testAddShardReplica with journal batch size 1 02:48:33,651 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true] 02:48:33,651 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 02:48:33,651 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddShardReplica 02:48:33,651 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: lastSequenceNumber prior to capture: 1 02:48:33,651 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:48:33,652 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating capture snapshot for new server member-2-shard-cars-oper_testAddShardReplica 02:48:33,652 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: CaptureSnapshotReply received by actor 02:48:33,652 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@626c3a27, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddShardReplica, ServerConfigPayload=null] 02:48:33,652 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 02:48:33,652 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendInstallSnapshot 02:48:33,652 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testAddShardReplica: 229 02:48:33,652 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:33,652 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:33,652 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 02:48:33,652 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddShardReplica is initialized 02:48:33,652 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:33,652 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddShardReplica is no longer the leader 02:48:33,652 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , received role change from null to Follower 02:48:33,652 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddShardReplica, member-2-shard-cars-oper_testAddShardReplica), Chunk: 1/1 02:48:33,652 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testAddShardReplica 02:48:33,652 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:48:33,652 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddShardReplica from null to Follower 02:48:33,652 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 1 02:48:33,653 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: SnapshotComplete received 02:48:33,653 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a snapshot response 02:48:33,653 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a journal response 02:48:33,653 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 02:48:33,653 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 02:48:33,653 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 02:48:33,653 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 02:48:33,653 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testAddShardReplica 02:48:33,653 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-oper_testAddShardReplica to state 02:48:33,653 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 02:48:33,653 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=-1] 02:48:33,654 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 02:48:33,654 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@74496be2, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 02:48:33,654 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 02:48:33,654 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 02:48:33,654 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:48:33,654 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 1 02:48:33,654 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to -1 02:48:33,654 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 02:48:33,654 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot 02:48:33,654 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 02:48:33,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#1712570718] with name member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher 02:48:33,655 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#1712570718] 02:48:33,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: state snapshot applied in 351.1 μs 02:48:33,655 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Done applying snapshot 02:48:33,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 02:48:33,655 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 02:48:33,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: SnapshotComplete received 02:48:33,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: handled a journal response 02:48:33,655 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 104.8 μs 02:48:33,655 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#1712570718] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:33,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: handled a snapshot response 02:48:33,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 02:48:33,656 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 02:48:33,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 02:48:33,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testAddShardReplica] 02:48:33,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]] 02:48:33,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:33,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:33,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=d20b6e6f-1df2-49b7-ab16-aab7bc0766f2}, logIndex: 0, isSendImmediate: true 02:48:33,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:33,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testAddShardReplica 02:48:33,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 02:48:33,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true] 02:48:33,657 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:33,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 02:48:33,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:33,657 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999 02:48:33,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:33,657 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 02:48:33,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 02:48:33,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:33,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddShardReplica 02:48:33,658 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 02:48:33,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 02:48:33,658 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 1 02:48:33,658 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:33,658 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 02:48:33,658 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 02:48:33,658 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 02:48:33,658 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 02:48:33,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:33,658 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#497785724] 02:48:33,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 02:48:33,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 02:48:33,658 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:33,658 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:33,658 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:33,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 02:48:33,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddShardReplica 02:48:33,659 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 0, nextIndex: 1 02:48:33,659 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:48:33,659 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 0 02:48:33,659 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 02:48:33,659 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 02:48:33,659 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddShardReplica is no longer the leader 02:48:33,659 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 02:48:33,659 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 02:48:33,659 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:33,659 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:33,659 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=0 02:48:33,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 0 02:48:33,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:33,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 0 02:48:33,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 02:48:33,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 02:48:33,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=0 02:48:33,660 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 02:48:33,666 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:48:33,676 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-3392037391089953488] 02:48:33,676 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 02:48:33,679 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:33,679 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 02:48:33,679 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:33,679 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:33,693 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:33,694 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:33,697 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 02:48:33,711 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1269974069] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:33,715 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 02:48:33,725 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 02:48:33,730 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 02:48:33,732 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:48:33,732 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:33,732 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 02:48:33,732 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:33,732 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 02:48:33,733 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 02:48:33,734 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6efbbe77 02:48:33,734 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} started 02:48:33,735 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:33,736 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:33,737 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:33,739 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 02:48:33,739 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#2129205216] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:33,740 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 02:48:33,741 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 02:48:33,741 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:33,741 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 02:48:33,741 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:33,741 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 02:48:33,743 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 02:48:33,744 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2fc6b44c 02:48:33,744 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} started 02:48:33,745 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:33,797 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:33,797 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 02:48:33,797 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 02:48:33,797 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 02:48:33,798 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:33,798 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 02:48:33,798 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 02:48:33,798 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 02:48:33,953 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 02:48:33,972 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 02:48:34,592 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 02:48:34,592 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:34,592 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 02:48:34,592 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 02:48:34,592 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 02:48:34,592 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:34,592 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 02:48:34,592 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 02:48:34,592 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 02:48:34,600 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:34,600 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 02:48:34,600 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:34,600 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 02:48:34,693 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 02:48:34,697 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:34,697 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 02:48:34,697 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 02:48:34,697 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 02:48:34,697 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:34,697 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 02:48:34,697 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 02:48:34,698 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 02:48:34,747 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 02:48:34,747 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 02:48:34,748 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:34,748 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica] 02:48:34,751 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 02:48:34,751 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187, primaryVersion=13] 02:48:34,754 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187 for shard member-3-shard-cars-config_testAddShardReplica 02:48:34,755 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-config_testAddShardReplica, persistent : true 02:48:34,755 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-2000396452] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:34,756 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#-108136908 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 02:48:34,756 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#369905864] 02:48:34,756 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#-108136908] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:34,756 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testAddShardReplica with journal batch size 1 02:48:34,756 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#369905864] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:34,757 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onAddServer: AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica, votingMember=true], state: Idle 02:48:34,757 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica, votingMember=true] 02:48:34,758 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testAddShardReplica 02:48:34,758 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: lastSequenceNumber prior to capture: 3 02:48:34,758 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:48:34,758 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating capture snapshot for new server member-3-shard-cars-config_testAddShardReplica 02:48:34,758 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: CaptureSnapshotReply received by actor 02:48:34,758 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:34,758 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:34,758 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1bb4ab3c, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddShardReplica, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:34,758 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 02:48:34,758 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendInstallSnapshot 02:48:34,758 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:34,759 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testAddShardReplica is no longer the leader 02:48:34,759 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 02:48:34,759 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): next snapshot chunk size for follower member-3-shard-cars-config_testAddShardReplica: 229 02:48:34,759 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 02:48:34,759 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 02:48:34,759 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddShardReplica, member-3-shard-cars-config_testAddShardReplica), Chunk: 1/1 02:48:34,759 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 02:48:34,759 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 3 02:48:34,759 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: SnapshotComplete received 02:48:34,759 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplica 02:48:34,759 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a snapshot response 02:48:34,759 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a journal response 02:48:34,759 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 02:48:34,760 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]" message is greater than follower's term 0 - updating term 02:48:34,760 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 02:48:34,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:34,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 02:48:34,761 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-config_testAddShardReplica 02:48:34,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-config_testAddShardReplica to state 02:48:34,761 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 02:48:34,762 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=-1] 02:48:34,762 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 02:48:34,762 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3c55d58d, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:34,762 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 02:48:34,762 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 02:48:34,762 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:48:34,762 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 1 02:48:34,762 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 02:48:34,762 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 02:48:34,762 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 02:48:34,762 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Applying snapshot 02:48:34,762 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 02:48:34,763 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-1454291919] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 02:48:34,763 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 375.3 μs 02:48:34,763 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Done applying snapshot 02:48:34,763 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 02:48:34,763 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-1454291919] 02:48:34,763 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 02:48:34,763 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 02:48:34,763 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 02:48:34,763 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-config_testAddShardReplica] 02:48:34,764 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-1454291919] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:34,764 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]] 02:48:34,764 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:34,764 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 02:48:34,764 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:34,764 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 99.67 μs 02:48:34,764 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=f51c9671-2f2d-46fd-8d74-b3832940a355}, logIndex: 1, isSendImmediate: true 02:48:34,764 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: SnapshotComplete received 02:48:34,764 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:34,764 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: handled a snapshot response 02:48:34,764 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testAddShardReplica 02:48:34,764 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: handled a journal response 02:48:34,764 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 02:48:34,764 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:34,764 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testAddShardReplica 02:48:34,764 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 02:48:34,764 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica, votingMember=true] 02:48:34,765 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:34,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 02:48:34,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:34,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:34,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 02:48:34,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:34,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 02:48:34,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 02:48:34,765 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:34,766 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:34,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187 02:48:34,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 02:48:34,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 02:48:34,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 02:48:34,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 1 02:48:34,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 02:48:34,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 02:48:34,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 02:48:34,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:34,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testAddShardReplica 02:48:34,766 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:34,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 02:48:34,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 02:48:34,766 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 02:48:34,767 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:34,767 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:34,767 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 02:48:34,767 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:34,767 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 02:48:34,767 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 1 02:48:34,767 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:34,767 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 02:48:34,767 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:34,767 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 02:48:34,767 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 1 02:48:34,767 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:48:34,767 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 1 02:48:34,767 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 02:48:34,767 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1269974069] 02:48:34,767 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 02:48:34,767 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 02:48:34,767 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 02:48:34,768 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 02:48:34,768 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 02:48:34,768 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 02:48:34,768 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 02:48:34,768 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 02:48:34,768 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:34,768 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddShardReplica 02:48:34,768 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-2000396452] 02:48:34,768 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 02:48:34,768 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testAddShardReplica is no longer the leader 02:48:34,768 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-2000396452] 02:48:34,768 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 02:48:34,768 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#2129205216] 02:48:34,768 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 02:48:34,768 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 02:48:34,768 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 02:48:34,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 02:48:34,769 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:34,769 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 02:48:34,769 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 1 02:48:34,769 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:48:34,769 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 1 02:48:34,769 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 02:48:34,769 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 02:48:34,769 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:34,769 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 02:48:34,769 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica] 02:48:34,769 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 02:48:34,770 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 02:48:34,770 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999, primaryVersion=13] 02:48:34,772 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999 for shard member-3-shard-cars-oper_testAddShardReplica 02:48:34,772 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Shard created : member-3-shard-cars-oper_testAddShardReplica, persistent : true 02:48:34,773 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testAddShardReplica with journal batch size 1 02:48:34,773 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-667116706] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:34,774 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#-883120307 created and ready for shard:member-3-shard-cars-oper_testAddShardReplica 02:48:34,774 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/shard-member-3:snapshot-read#-1165848895] 02:48:34,774 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#-883120307] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:34,774 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/shard-member-3:snapshot-read#-1165848895] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:34,774 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:34,774 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:34,774 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:34,775 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testAddShardReplica is no longer the leader 02:48:34,775 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , received role change from null to Follower 02:48:34,775 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 02:48:34,775 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testAddShardReplica is initialized 02:48:34,775 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testAddShardReplica 02:48:34,775 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onAddServer: AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica, votingMember=true], state: Idle 02:48:34,775 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testAddShardReplica from null to Follower 02:48:34,775 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica, votingMember=true] 02:48:34,775 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testAddShardReplica 02:48:34,775 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: lastSequenceNumber prior to capture: 3 02:48:34,775 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 02:48:34,775 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating capture snapshot for new server member-3-shard-cars-oper_testAddShardReplica 02:48:34,775 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: CaptureSnapshotReply received by actor 02:48:34,775 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7c66929b, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddShardReplica, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:34,775 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 02:48:34,775 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendInstallSnapshot 02:48:34,775 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): next snapshot chunk size for follower member-3-shard-cars-oper_testAddShardReplica: 229 02:48:34,775 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddShardReplica, member-3-shard-cars-oper_testAddShardReplica), Chunk: 1/1 02:48:34,776 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 02:48:34,776 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 3 02:48:34,776 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: SnapshotComplete received 02:48:34,776 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a snapshot response 02:48:34,776 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a journal response 02:48:34,776 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]" message is greater than follower's term 0 - updating term 02:48:34,777 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 02:48:34,777 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:34,777 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 02:48:34,777 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testAddShardReplica 02:48:34,777 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Converted InstallSnapshot from leader: member-1-shard-cars-oper_testAddShardReplica to state 02:48:34,777 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 02:48:34,777 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=-1] 02:48:34,777 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 02:48:34,778 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@51af61f7, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:34,778 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 02:48:34,778 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 02:48:34,778 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 02:48:34,778 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 1 02:48:34,778 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 02:48:34,778 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 02:48:34,778 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 02:48:34,778 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot 02:48:34,778 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 02:48:34,778 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#1652744598] with name member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher 02:48:34,778 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#1652744598] 02:48:34,778 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: state snapshot applied in 350.5 μs 02:48:34,778 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Done applying snapshot 02:48:34,778 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 02:48:34,778 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 02:48:34,779 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 90.72 μs 02:48:34,779 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#1652744598] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:34,779 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: SnapshotComplete received 02:48:34,779 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: handled a snapshot response 02:48:34,779 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: handled a journal response 02:48:34,779 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 02:48:34,779 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 02:48:34,779 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 02:48:34,779 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-oper_testAddShardReplica] 02:48:34,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]] 02:48:34,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:34,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:34,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=5163659d-0a20-47d8-9b6e-daa1f6c5ecd0}, logIndex: 1, isSendImmediate: true 02:48:34,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:34,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testAddShardReplica 02:48:34,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 02:48:34,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:34,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:34,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 02:48:34,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica, votingMember=true] 02:48:34,781 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:34,781 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 02:48:34,781 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:34,781 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:34,781 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 02:48:34,781 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:34,781 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 02:48:34,781 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 02:48:34,781 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:34,782 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:34,782 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 02:48:34,782 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 02:48:34,782 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 1 02:48:34,782 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 02:48:34,782 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 02:48:34,782 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 02:48:34,782 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:34,782 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999 02:48:34,782 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 02:48:34,782 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:34,782 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 02:48:34,782 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testAddShardReplica 02:48:34,782 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 02:48:34,782 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 02:48:34,783 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:34,783 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:34,783 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 02:48:34,783 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:34,783 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 02:48:34,783 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#2129205216] 02:48:34,783 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 02:48:34,783 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 1 02:48:34,783 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:34,783 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 02:48:34,783 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:48:34,783 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 02:48:34,783 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 02:48:34,783 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddShardReplica 02:48:34,783 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 02:48:34,783 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testAddShardReplica is no longer the leader 02:48:34,783 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 02:48:34,784 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 02:48:34,784 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 1 02:48:34,784 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 02:48:34,784 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:48:34,784 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 02:48:34,784 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 1 02:48:34,784 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 02:48:34,784 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 02:48:34,784 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 02:48:34,784 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 02:48:34,784 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 02:48:34,784 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-667116706] 02:48:34,784 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-667116706] 02:48:34,784 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1105992632] 02:48:34,784 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 02:48:34,784 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 02:48:34,784 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:34,784 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1728454553] 02:48:34,784 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1728454553] 02:48:34,785 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#497785724] 02:48:34,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 02:48:34,785 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:34,785 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:34,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 1 02:48:34,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 02:48:34,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 1 02:48:34,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 02:48:34,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 02:48:34,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 02:48:34,785 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 02:48:34,835 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1105992632] 02:48:34,835 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:34,835 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 02:48:34,836 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 02:48:34,847 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 02:48:34,858 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=75612, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 02:48:34,866 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 02:48:34,869 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 02:48:34,896 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 28.97 ms 02:48:34,899 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,919 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: readying transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,919 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 ready 02:48:34,924 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: transitioned member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 to ready 02:48:34,925 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 02:48:34,925 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Validating transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,925 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 validated 02:48:34,925 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 02:48:34,925 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 02:48:34,926 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Preparing transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,929 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 candidate ready 02:48:34,930 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 02:48:34,933 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 02:48:34,933 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 02:48:34,933 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 prepared 02:48:34,933 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 02:48:34,933 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 02:48:34,934 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Starting commit for transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,940 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 02:48:34,940 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 02:48:34,940 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 02:48:34,940 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, logIndex: 2, isSendImmediate: true 02:48:34,940 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:48:34,940 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testAddShardReplica 02:48:34,942 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 02:48:34,942 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:48:34,942 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testAddShardReplica 02:48:34,942 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 02:48:34,942 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 02:48:34,945 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 02:48:34,945 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 02:48:34,946 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:34,946 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:48:34,946 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log Simple{size=235} 02:48:34,946 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 02:48:34,946 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 02:48:34,946 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:34,946 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 02:48:34,946 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:34,946 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:48:34,946 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log Simple{size=235} 02:48:34,946 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 02:48:34,946 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 02:48:34,946 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:34,946 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 02:48:34,946 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 02:48:34,946 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 02:48:34,947 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 02:48:34,947 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 02:48:34,947 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 2 02:48:34,947 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 02:48:34,947 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 02:48:34,948 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Resuming commit of transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,950 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-config_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@45fb489d}, purged=MutableUnsignedLongSet{size=0}} 02:48:34,950 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:34,951 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,951 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 02:48:34,951 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 02:48:34,957 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-config_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#657937249] with name member-1-shard-cars-config_testAddShardReplica-DTCL-publisher 02:48:34,957 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#657937249] 02:48:34,958 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 02:48:34,958 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-config_testAddShardReplica-DTCL-publisher: 116.8 μs 02:48:34,958 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#657937249] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:34,961 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1269974069] 02:48:34,961 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:34,961 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 02:48:34,962 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 02:48:34,973 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 02:48:34,973 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:48:34,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 02:48:34,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:48:34,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 02:48:34,975 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 2 02:48:34,975 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 02:48:34,975 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 2 02:48:34,975 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 02:48:34,975 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 02:48:34,975 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,978 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@31aa46d5}, purged=MutableUnsignedLongSet{size=0}} 02:48:34,978 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:34,978 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,978 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,978 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 02:48:34,978 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 02:48:34,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 2 02:48:34,978 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 02:48:34,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 02:48:34,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 2 02:48:34,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 02:48:34,978 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 02:48:34,978 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 115.5 μs 02:48:34,979 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,979 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@57c6899b}, purged=MutableUnsignedLongSet{size=0}} 02:48:34,979 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:34,979 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,979 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 02:48:34,979 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 02:48:34,979 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 02:48:34,979 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 158.8 μs 02:48:34,984 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 02:48:34,984 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 02:48:34,985 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, logIndex: 3, isSendImmediate: false 02:48:34,985 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-config_testAddShardReplica 02:48:34,985 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 02:48:34,985 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-config_testAddShardReplica 02:48:34,985 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 02:48:34,987 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 02:48:34,987 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=10910, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 02:48:34,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 02:48:34,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:34,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:48:34,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 02:48:34,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 02:48:34,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 02:48:34,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:34,987 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 02:48:34,987 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 02:48:34,987 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 02:48:34,987 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:34,987 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:48:34,987 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 02:48:34,988 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 02:48:34,988 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 02:48:34,988 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:34,988 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 02:48:34,988 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 02:48:34,988 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 02:48:34,988 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 3 02:48:34,988 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 02:48:34,988 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 02:48:34,988 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 02:48:34,988 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: replication of member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007ff108942378@5e6ba65b 02:48:34,989 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 660.1 μs 02:48:34,989 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 02:48:34,989 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: finished purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,991 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:34,991 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,991 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 02:48:34,991 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:48:34,992 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 02:48:34,992 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:48:34,992 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 3 02:48:34,992 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 02:48:34,992 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 3 02:48:34,992 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 02:48:34,992 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 02:48:34,992 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:34,992 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,992 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 02:48:34,992 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 02:48:34,993 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 02:48:34,993 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,993 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 3 02:48:34,993 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 02:48:34,993 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 3 02:48:34,993 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 02:48:34,993 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 02:48:34,993 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:34,993 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:34,993 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 02:48:34,993 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 02:48:35,0 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#2129205216] 02:48:35,0 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:35,0 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 02:48:35,0 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 02:48:35,1 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 02:48:35,1 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=12610, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 02:48:35,2 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 02:48:35,2 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 02:48:35,3 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 624.2 μs 02:48:35,4 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,4 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: readying transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,4 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 ready 02:48:35,4 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: transitioned member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 to ready 02:48:35,4 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 02:48:35,4 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Validating transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,4 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 validated 02:48:35,4 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 02:48:35,4 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 02:48:35,4 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Preparing transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,4 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 candidate ready 02:48:35,4 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 prepared 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Starting commit for transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 233 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, logIndex: 2, isSendImmediate: true 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testAddShardReplica 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 02:48:35,5 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 02:48:35,6 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 02:48:35,6 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:35,6 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:48:35,6 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log Simple{size=233} 02:48:35,6 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 02:48:35,6 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 02:48:35,6 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 02:48:35,6 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:35,6 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 02:48:35,6 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:35,6 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:48:35,6 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log Simple{size=233} 02:48:35,6 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 02:48:35,7 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 02:48:35,7 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:35,7 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 02:48:35,7 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 02:48:35,7 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 02:48:35,7 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 02:48:35,7 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 02:48:35,7 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 2 02:48:35,7 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 02:48:35,7 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 02:48:35,7 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Resuming commit of transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,7 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@13caacf0}, purged=MutableUnsignedLongSet{size=0}} 02:48:35,7 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:35,7 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,7 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 02:48:35,7 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 02:48:35,8 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1763126619] with name member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher 02:48:35,8 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 02:48:35,8 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:48:35,8 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 02:48:35,8 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1763126619] 02:48:35,8 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 02:48:35,8 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher: 56.85 μs 02:48:35,9 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1763126619] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:35,9 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 02:48:35,9 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 02:48:35,9 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 2 02:48:35,9 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 02:48:35,9 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 2 02:48:35,9 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 02:48:35,9 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 02:48:35,9 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#497785724] 02:48:35,9 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@78a857f6}, purged=MutableUnsignedLongSet{size=0}} 02:48:35,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:35,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,10 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 02:48:35,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 02:48:35,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 02:48:35,10 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 168.5 μs 02:48:35,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:35,10 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 02:48:35,11 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,11 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:35,11 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:35,11 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, logIndex: 3, isSendImmediate: false 02:48:35,11 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 2 02:48:35,11 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 02:48:35,11 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 2 02:48:35,11 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 02:48:35,11 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 02:48:35,11 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 02:48:35,11 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:35,11 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,11 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@65966e06}, purged=MutableUnsignedLongSet{size=0}} 02:48:35,11 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:35,11 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,11 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 02:48:35,12 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 62.08 μs 02:48:35,12 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 02:48:35,12 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 02:48:35,12 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:35,12 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:35,12 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 02:48:35,12 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=9140, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 02:48:35,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:35,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:35,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:48:35,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 02:48:35,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:35,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 02:48:35,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:35,13 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 02:48:35,13 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 02:48:35,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:35,13 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 347.7 μs 02:48:35,14 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open snapshot member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,14 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 02:48:35,14 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 02:48:35,14 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 3 02:48:35,14 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 02:48:35,15 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 02:48:35,15 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007ff108942378@59c9e373 02:48:35,15 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,15 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:35,15 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,15 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 02:48:35,15 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:48:35,15 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 02:48:35,15 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 3 02:48:35,15 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 02:48:35,16 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 3 02:48:35,16 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 02:48:35,16 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 02:48:35,16 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:35,16 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,16 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 02:48:35,16 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 02:48:35,16 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1994489659] 02:48:35,16 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:35,16 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187] 02:48:35,17 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187] 02:48:35,17 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-787511874] 02:48:35,17 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 02:48:35,17 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999] 02:48:35,17 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999] 02:48:35,17 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1105992632] 02:48:35,17 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:35,17 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1728454553] 02:48:35,17 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1728454553] 02:48:35,18 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#497785724] 02:48:35,18 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 02:48:35,18 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:35,18 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:35,68 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:35,118 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:35,169 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:35,220 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:35,270 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:35,321 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:35,371 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:35,422 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:35,472 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:35,523 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:35,571 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-oper_testAddShardReplica 02:48:35,571 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:35,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:35,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:35,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 02:48:35,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 02:48:35,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:35,573 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1453053178] 02:48:35,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 02:48:35,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 3 02:48:35,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:35,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 02:48:35,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 3 02:48:35,573 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 02:48:35,574 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 02:48:35,574 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:35,574 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,574 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 02:48:35,574 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 02:48:35,574 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:48:35,575 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:35,575 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 02:48:35,575 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1269974069] 02:48:35,575 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:35,576 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-2000396452] 02:48:35,576 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-2000396452] 02:48:35,576 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#2129205216] 02:48:35,576 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 02:48:35,576 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-667116706] 02:48:35,576 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-667116706] 02:48:35,577 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 02:48:35,577 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 02:48:35,577 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 02:48:35,577 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onShutDown 02:48:35,579 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node [akka://cluster-test@127.0.0.1:2557] as [Down] 02:48:35,579 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testAddShardReplica 02:48:35,579 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 02:48:35,580 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 02:48:35,580 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 02:48:35,581 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 02:48:35,581 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testAddShardReplica 02:48:35,581 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 02:48:35,581 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onShutDown 02:48:35,582 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testAddShardReplica 02:48:35,582 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Aborting 0 pending queued transactions 02:48:35,582 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 02:48:35,582 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 02:48:35,584 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:35,584 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:35,585 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Received gossip where this member has been downed, from [akka://cluster-test@127.0.0.1:2558] 02:48:35,585 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Node has been marked as DOWN. Shutting down myself 02:48:35,588 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Shutting down... 02:48:35,588 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Successfully shut down 02:48:35,594 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:35,594 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:35,594 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:35,595 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:35,597 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:48:35,624 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:48:35,635 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5115377124146372083] 02:48:35,635 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 02:48:35,639 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Registered cluster JMX MBean [akka:type=Cluster] 02:48:35,639 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 02:48:35,642 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:35,642 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:35,652 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:35,654 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:35,661 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 02:48:35,668 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-0-txn-0-0} 02:48:35,668 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-0} 02:48:35,671 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 02:48:35,673 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: aborting transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,673 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1497042349] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:35,673 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onSnapshotOffer: ShardManagerSnapshot [ShardList = [cars] ] 02:48:35,674 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: adding shard cars 02:48:35,674 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:35,674 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:35,674 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, logIndex: 4, isSendImmediate: false 02:48:35,674 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 02:48:35,674 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testAddShardReplica 02:48:35,676 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:35,679 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} started 02:48:35,680 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6ac8c47b 02:48:35,680 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:35,680 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testAddShardReplica 02:48:35,680 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1497042349] 02:48:35,681 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:35,681 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 02:48:35,682 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Shard created : member-3-shard-cars-config_testAddShardReplica, persistent : true 02:48:35,682 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-406209629] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:35,682 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#1381501269] 02:48:35,682 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#-1020518653 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 02:48:35,682 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#-1020518653] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:35,682 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#1381501269] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:35,682 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testAddShardReplica with journal batch size 1 02:48:35,683 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.SnapshotOffer from Actor[akka://cluster-test/system/in-memory-snapshot-store#-1636015801] 02:48:35,683 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: SnapshotOffer called. 02:48:35,683 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 02:48:35,683 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 02:48:35,683 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying recovered snapshot 02:48:35,688 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 02:48:35,690 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#428443819] 02:48:35,690 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#428443819] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 02:48:35,690 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 02:48:35,690 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#428443819] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:35,690 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 60.10 μs 02:48:35,690 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 2.261 ms 02:48:35,690 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING]] 02:48:35,690 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Recovery snapshot applied for member-3-shard-cars-config_testAddShardReplica in 7.459 ms: snapshotIndex=0, snapshotTerm=1, journal-size=0 02:48:35,691 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:35,691 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 1, size: 267 02:48:35,691 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING]] 02:48:35,691 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 02:48:35,691 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 1 to 1 02:48:35,691 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 02:48:35,691 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:35,691 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 2, size: 235 02:48:35,691 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 02:48:35,691 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 2 to 2 02:48:35,691 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 02:48:35,696 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1a7b9ade}, purged=MutableUnsignedLongSet{size=0}} 02:48:35,696 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:35,696 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:35,696 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 02:48:35,696 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:35,696 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 3, size: 56 02:48:35,696 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 02:48:35,696 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 3 to 3 02:48:35,696 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 02:48:35,696 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:35,696 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 02:48:35,696 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 02:48:35,697 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:35,697 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Recovery completed in 13.67 ms - Switching actor to Follower - last log index = 3, last log term = 1, snapshot index = 0, snapshot term = 1, journal size = 3 02:48:35,697 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 02:48:35,697 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 02:48:35,697 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 02:48:35,697 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-406209629] 02:48:35,697 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-406209629] 02:48:35,699 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:48:35,699 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testAddShardReplica is no longer the leader 02:48:35,699 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 02:48:35,700 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplica 02:48:35,700 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1497042349] 02:48:35,700 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 02:48:35,700 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:48:35,700 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 02:48:35,890 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-oper_testAddShardReplica 02:48:35,891 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:35,891 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:35,891 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:35,892 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:35,893 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:35,893 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 02:48:35,893 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 02:48:35,893 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:35,893 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 02:48:35,893 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=4, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:35,893 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:35,894 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 4, nextIndex: 5 02:48:35,894 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 02:48:35,894 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 4 02:48:35,894 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 3 to 4 02:48:35,894 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 4 data AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 02:48:35,894 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda/0x00007ff10894d528@21645ea4 02:48:35,895 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@16bd85b}, purged=MutableUnsignedLongSet{size=0}} 02:48:35,895 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:35,895 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Aborted transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,895 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 3 for term 1 02:48:35,895 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 02:48:35,895 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=4 02:48:35,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 4 02:48:35,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 02:48:35,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 4 02:48:35,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 3 to 4 02:48:35,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 4 data AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 02:48:35,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1d12cf4c}, purged=MutableUnsignedLongSet{size=0}} 02:48:35,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:35,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Aborted transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 3 for term 1 02:48:35,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=4 02:48:35,900 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:35,900 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:35,900 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:35,901 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, logIndex: 5, isSendImmediate: false 02:48:35,901 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:36,210 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-oper_testAddShardReplica 02:48:36,210 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=4, prevLogTerm=1, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:36,210 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:36,210 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:36,212 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=4, prevLogTerm=1, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:36,213 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 02:48:36,213 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 02:48:36,213 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 02:48:36,213 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:36,213 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 02:48:36,213 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=5, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:36,214 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 02:48:36,215 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 5, nextIndex: 6 02:48:36,215 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 02:48:36,216 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 5 02:48:36,216 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 4 to 5 02:48:36,216 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 02:48:36,216 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007ff108942378@67d4d106 02:48:36,216 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:36,216 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:36,216 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:36,216 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 3 02:48:36,216 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=5 02:48:36,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 5 02:48:36,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 02:48:36,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 5 02:48:36,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 4 to 5 02:48:36,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 02:48:36,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 02:48:36,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 02:48:36,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=5 02:48:36,530 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:36,530 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:36,601 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-3392037391089953488] but current is [-5115377124146372083]. Reason: ActorSystem terminated 02:48:36,601 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-3392037391089953488] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:48:36,665 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 02:48:36,666 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 02:48:36,666 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999] to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:36,666 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 02:48:36,666 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = null 02:48:36,666 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 02:48:36,666 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 02:48:36,666 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 02:48:36,667 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999] to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:36,667 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999] to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:36,667 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 02:48:36,667 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 02:48:36,823 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#-440947785]] intended for system uid [-3392037391089953488], self uid is [-5115377124146372083] 02:48:36,851 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:36,851 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:36,853 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999] to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:37,171 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:37,171 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:37,174 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#571000999] to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:37,183 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 02:48:37,183 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testAddShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 02:48:37,490 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:37,491 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:37,682 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 02:48:37,810 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:37,810 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:38,130 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:38,130 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:38,451 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:38,451 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:38,702 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 02:48:38,770 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:38,771 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:38,843 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#-440947785]] intended for system uid [-3392037391089953488], self uid is [-5115377124146372083] 02:48:39,90 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:39,90 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:39,410 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:39,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:39,723 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 02:48:39,730 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:39,730 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:40,51 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:40,51 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:40,370 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:40,370 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:40,662 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#-302558472]] intended for system uid [-3392037391089953488], self uid is [-5115377124146372083] 02:48:40,690 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:40,690 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:40,711 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Down)]. 02:48:40,712 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:40,712 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:40,742 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 02:48:40,933 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Down)]. 02:48:40,933 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:40,933 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:41,10 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:41,10 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:41,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:41,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:41,651 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:41,651 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:41,732 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2557] 02:48:41,733 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:41,733 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:41,733 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-3392037391089953488] but current is [-5115377124146372083]. Reason: Cluster member removed, previous status [Down] 02:48:41,763 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 02:48:41,767 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:48:41,768 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:41,768 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 02:48:41,768 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 02:48:41,768 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is ready - notifying 1 OnShardInitialized callbacks 02:48:41,768 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 02:48:41,768 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 02:48:41,768 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:41,768 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 02:48:41,768 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 02:48:41,768 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 02:48:41,768 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testAddShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 02:48:41,769 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testAddShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 02:48:41,769 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 02:48:41,772 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: retiring state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=6779307159, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 02:48:41,775 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, nanosAgo=19600, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 02:48:41,776 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 02:48:41,776 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 02:48:41,776 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1511667187], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 372.9 μs 02:48:41,778 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-1-txn-0-1 02:48:41,780 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 02:48:41,781 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 02:48:41,781 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 02:48:41,781 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onShutDown 02:48:41,781 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testAddShardReplica 02:48:41,781 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 02:48:41,782 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 02:48:41,782 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 02:48:41,784 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:41,784 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:41,791 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:41,791 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:41,792 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:41,792 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:41,794 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:48:41,811 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 02:48:41,812 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddShardReplica 02:48:41,812 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 02:48:41,812 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onShutDown 02:48:41,813 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testAddShardReplica 02:48:41,813 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 02:48:41,813 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 02:48:41,813 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 02:48:41,813 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 02:48:41,814 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddShardReplica 02:48:41,814 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 02:48:41,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onShutDown 02:48:41,814 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testAddShardReplica 02:48:41,814 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Aborting 0 pending queued transactions 02:48:41,814 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 02:48:41,814 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 02:48:41,815 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:41,815 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:41,959 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-3392037391089953488] but current is [-5115377124146372083]. Reason: Cluster member removed, previous status [Down] 02:48:41,970 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:41,970 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:42,290 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:42,290 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:42,610 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:42,610 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:42,752 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 02:48:42,752 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:42,753 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 02:48:42,753 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 02:48:42,753 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 02:48:42,753 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:42,753 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testAddShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 02:48:42,753 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 02:48:42,753 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 02:48:42,753 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 02:48:42,753 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testAddShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 02:48:42,800 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-5115377124146372083] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:48:42,830 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4656462791534312841] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:48:42,833 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:42,834 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:42,836 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:48:42.838] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:42,844 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 02:48:42,844 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplica 02:48:42,844 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 02:48:42,844 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onShutDown 02:48:42,844 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating leader transfer 02:48:42,844 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplica, leaderId=null, leaderPayloadVersion=13] 02:48:42,844 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@53f6be5d 02:48:42,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@53f6be5d 02:48:42,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: pauseLeader successfully completed - doing transfer 02:48:42,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Attempting to transfer leadership 02:48:42,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:48:42,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 02:48:42,930 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:42,930 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:43,250 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:43,251 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:43,570 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:43,570 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:43,771 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 02:48:43,890 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:43,890 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:44,211 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:44,211 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:44,530 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 4 is present for follower member-3-shard-cars-oper_testAddShardReplica 02:48:44,530 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=3, prevLogTerm=1, leaderCommit=5, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=1, payload=AbortTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}], SimpleReplicatedLogEntry [index=5, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 02:48:47,851 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2559, Up)]. 02:48:47,851 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:47,851 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:49,657 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:48:49,891 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 02:48:49,891 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:48:49,892 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:49,892 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:50,296 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:48:51,900 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: leader transfer aborted 02:48:51,900 PM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: Failed to transfer leadership in 9.056 s 02:48:51,900 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In unpauseLeader 02:48:51,902 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: leader transfer failed - sending PoisonPill 02:48:51,902 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Leadership transfer expired 02:48:51,902 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddShardReplica 02:48:51,903 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 02:48:51,903 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:48:51,903 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:48:51,903 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 02:48:51,904 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplica 02:48:51,904 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onShutDown 02:48:51,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating leader transfer 02:48:51,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1d7138f9 02:48:51,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1d7138f9 02:48:51,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: pauseLeader successfully completed - doing transfer 02:48:51,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Attempting to transfer leadership 02:48:51,905 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 02:48:51,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 3 02:48:51,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 5, leaderSnapShotIndex: 3 02:48:51,905 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddShardReplica, leaderId=null, leaderPayloadVersion=13] 02:48:53,170 PM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testAddShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 02:48:53,170 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testAddShardReplica, member-2-shard-cars-oper_testAddShardReplica] 02:48:53,171 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 5, leaderSnapShotIndex: 3 02:48:53,171 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 3 02:48:53,171 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 02:48:53,171 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: leader transfer aborted 02:48:53,171 PM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: Failed to transfer leadership in 1.266 s 02:48:53,171 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In unpauseLeader 02:48:53,171 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: leader transfer failed - sending PoisonPill 02:48:53,171 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:48:53,172 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddShardReplica 02:48:53,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Aborting 0 pending queued transactions 02:48:53,172 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:48:53,172 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:48:53,174 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:53,174 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:53,177 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:48:53.180] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:53,200 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:48:53,209 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-7928532264803239779] 02:48:53,211 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:48:53,211 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:48:53,211 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:48:53,212 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:53,212 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:53,227 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:53,228 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:48:53,228 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:48:53,229 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:48:53,229 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:53,233 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 02:48:53,246 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#65172221] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,247 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:48:53,249 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:48:53,249 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,249 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,249 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:53,249 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,249 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,250 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:53,251 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4f583062 02:48:53,251 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 02:48:53,252 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:53,252 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,252 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:53,255 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:53,256 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1608293555] 02:48:53,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,256 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 02:48:53,257 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 02:48:53,258 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 02:48:53,260 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,260 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1097697997 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,260 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,260 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1994637437] 02:48:53,260 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1608293555] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,261 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:53,261 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1097697997] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,261 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1115559355] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,261 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:53,261 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#351306427] 02:48:53,261 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:48:53,261 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 02:48:53,261 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 02:48:53,261 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#351306427] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,261 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,261 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,261 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 02:48:53,262 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 02:48:53,262 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 02:48:53,264 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 02:48:53,264 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1994637437] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,265 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#724363909 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,265 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#382626800] 02:48:53,265 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#724363909] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,265 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#382626800] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,265 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 02:48:53,266 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:48:53,266 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,266 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:53,266 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:53,267 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 02:48:53,267 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 02:48:53,267 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,267 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,267 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,267 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 02:48:53,267 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:53,267 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,267 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,268 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 02:48:53,268 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 02:48:53,269 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:53,270 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 02:48:53,271 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4a1105e9 02:48:53,271 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:53,271 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,272 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,272 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,272 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1432667645] 02:48:53,272 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,273 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,273 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,273 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#1651316941] 02:48:53,273 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 02:48:53,273 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 02:48:53,274 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 02:48:53,274 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#412227722 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,274 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#772622459] 02:48:53,274 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1432667645] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,274 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#1651316941] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,274 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#599527542] 02:48:53,275 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-1668921938 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,276 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:53,276 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 02:48:53,276 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:53,276 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 02:48:53,276 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 02:48:53,276 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,276 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,276 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 02:48:53,276 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 02:48:53,276 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 02:48:53,276 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:53,276 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:53,277 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,277 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,277 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 02:48:53,277 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 02:48:53,277 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 02:48:53,277 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 02:48:53,277 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 02:48:53,278 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:48:53,289 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [6857785753816278512] 02:48:53,289 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:48:53,297 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:53,297 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:48:53,297 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:53,297 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:53,304 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:53,311 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:53,313 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 02:48:53,326 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1721755042] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,326 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-1-txn-0-0} 02:48:53,332 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 02:48:53,343 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 02:48:53,343 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,344 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,347 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:53,349 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@46752613 02:48:53,350 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 02:48:53,350 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:53,350 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,351 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:53,351 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,352 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 02:48:53,352 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#1313655574] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,352 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1434923667 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,352 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1434923667] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,352 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 02:48:53,353 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 02:48:53,353 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-2067189081 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,353 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-2144609486] 02:48:53,354 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:53,360 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 02:48:53,361 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 02:48:53,362 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:48:53,367 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:48:53,368 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#1354254925] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,368 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-2067189081] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,368 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-2144609486] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,368 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#509117034] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,369 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-1224793300] 02:48:53,369 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-1224793300] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,370 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:53,370 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,370 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,370 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,370 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,370 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 02:48:53,370 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:53,370 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:53,370 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 02:48:53,370 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 02:48:53,371 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,371 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,371 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 02:48:53,371 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:53,371 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,371 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:53,371 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 02:48:53,371 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 02:48:53,371 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 02:48:53,371 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 02:48:53,372 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 02:48:53,372 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,372 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,372 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,372 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 02:48:53,372 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,372 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 02:48:53,372 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 02:48:53,373 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,373 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:53,373 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,373 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,375 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:53,376 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 02:48:53,381 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@f6216ba 02:48:53,381 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:53,381 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,382 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,382 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,382 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1888490411] 02:48:53,382 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,382 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Shard created : member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 02:48:53,382 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1888490411] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,383 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1268535178 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,383 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-180587524] 02:48:53,384 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,384 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1268535178] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,384 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,384 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1773877848] 02:48:53,384 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 02:48:53,384 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 02:48:53,385 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-753106401 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,385 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 02:48:53,385 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#1525953246] 02:48:53,385 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:53,385 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:53,385 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:53,385 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 02:48:53,385 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:53,385 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 02:48:53,385 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 02:48:53,385 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 02:48:53,385 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,385 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,385 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 02:48:53,385 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,385 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,386 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 02:48:53,386 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,386 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 02:48:53,386 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,386 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 02:48:53,386 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 02:48:53,386 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 02:48:53,388 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:48:53,396 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-9082138879440525737] 02:48:53,397 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 02:48:53,399 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:53,399 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 02:48:53,400 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:53,400 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:53,434 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:53,436 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:53,439 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 02:48:53,444 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1666689734] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,447 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 02:48:53,464 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 02:48:53,464 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,465 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,467 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:53,468 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3f81b5dc 02:48:53,469 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 02:48:53,469 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:53,470 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:53,470 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,471 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:53,472 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 02:48:53,473 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 02:48:53,473 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,474 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 02:48:53,475 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 02:48:53,475 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 02:48:53,476 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 02:48:53,476 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 02:48:53,476 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:48:53,476 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1217227812] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,476 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#200891389] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,476 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1498472954] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,477 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-2109137076 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,477 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#956057479] 02:48:53,477 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#566232995] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,477 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-2109137076] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,477 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#956057479] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,480 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:53,480 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:53,481 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,481 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,483 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 02:48:53,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 02:48:53,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 02:48:53,485 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:53,485 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,485 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,485 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,485 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,485 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#566232995 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,485 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 02:48:53,486 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#1676511073] 02:48:53,486 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#1676511073] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,487 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:53,487 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,487 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,488 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:53,489 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@335a27d2 02:48:53,489 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 02:48:53,489 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:53,489 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,489 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:53,490 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,490 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,490 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1459473491] 02:48:53,490 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,491 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,491 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,491 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1060178595] 02:48:53,492 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,492 PM [cluster-test-akka.actor.default-dispatcher-35] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 02:48:53,492 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:53,492 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:53,492 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 02:48:53,492 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 02:48:53,492 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 02:48:53,492 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,492 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,492 PM [cluster-test-akka.actor.default-dispatcher-35] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 02:48:53,492 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,492 PM [cluster-test-akka.actor.default-dispatcher-35] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 02:48:53,492 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 02:48:53,492 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 02:48:53,492 PM [cluster-test-akka.actor.default-dispatcher-35] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1459473491] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,493 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#-1896395431] 02:48:53,493 PM [cluster-test-akka.actor.default-dispatcher-35] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1999159439 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,493 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1999159439] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:53,493 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 02:48:53,494 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 02:48:53,495 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 02:48:53,497 PM [cluster-test-akka.actor.default-dispatcher-35] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#-1521679383] 02:48:53,497 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1501050817 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,497 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:53,497 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:53,497 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 02:48:53,497 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 02:48:53,497 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,497 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,497 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,497 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 02:48:53,497 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:53,497 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:53,497 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 02:48:53,497 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 02:48:53,497 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 02:48:53,497 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 02:48:53,497 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,497 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,497 PM [cluster-test-akka.actor.default-dispatcher-35] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 02:48:53,497 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,498 PM [cluster-test-akka.actor.default-dispatcher-35] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 02:48:53,498 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 02:48:53,598 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:48:53,598 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 02:48:53,598 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards] 02:48:53,598 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:53,598 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 02:48:53,599 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:53,599 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,599 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,599 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 02:48:53,600 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 02:48:53,647 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:48:53,648 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 02:48:53,648 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards] 02:48:53,648 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:53,648 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 02:48:53,648 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:53,648 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,648 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,648 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 02:48:53,648 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 02:48:53,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:48:53,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 02:48:53,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards] 02:48:53,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,658 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 02:48:53,658 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:53,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,658 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 02:48:53,658 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 02:48:53,678 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:48:53,678 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 02:48:53,678 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards] 02:48:53,678 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:53,678 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 02:48:53,678 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:53,678 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 02:48:53,678 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 02:48:53,678 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 02:48:53,678 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 02:48:53,997 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 02:48:53,998 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:53,998 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 02:48:54,8 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 02:48:54,8 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,8 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 02:48:54,38 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 02:48:54,38 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,38 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 02:48:54,58 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 02:48:54,58 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,58 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 02:48:54,229 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 02:48:54,229 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:48:54,229 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:54,229 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:54,229 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:54,229 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:54,229 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,229 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,236 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:54,236 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,236 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,236 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,236 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,237 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:54,322 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:54,322 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:54,322 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:54,322 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,322 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,357 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 02:48:54,357 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,357 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 02:48:54,358 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,358 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,359 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:54,359 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:54,359 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 02:48:54,359 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 02:48:54,359 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:54,360 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:54,360 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:54,360 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 02:48:54,360 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:54,360 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 02:48:54,360 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:54,360 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,360 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:54,360 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:54,360 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,360 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:54,361 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:48:54,361 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:48:54,361 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards] 02:48:54,361 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,361 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,361 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:48:54,361 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 02:48:54,362 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 02:48:54,362 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 02:48:54,362 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 02:48:54,362 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 02:48:54,362 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,363 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 02:48:54,363 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 02:48:54,363 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,363 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 02:48:54,363 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 02:48:54,363 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 02:48:54,363 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,363 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 02:48:54,364 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,411 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 02:48:54,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 02:48:54,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,418 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 02:48:54,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,419 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:54,419 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:54,419 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 02:48:54,419 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 02:48:54,419 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:54,419 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:54,419 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:54,419 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:54,419 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 02:48:54,419 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 02:48:54,419 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:54,419 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:54,419 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,419 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,419 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:54,419 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:54,420 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:48:54,420 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:48:54,421 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards] 02:48:54,421 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,421 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,421 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:48:54,421 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 02:48:54,421 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 02:48:54,421 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 02:48:54,421 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 02:48:54,422 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 02:48:54,422 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,422 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 02:48:54,422 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 02:48:54,422 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,422 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 02:48:54,423 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 02:48:54,423 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 02:48:54,423 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,423 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 02:48:54,423 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,458 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 02:48:54,458 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,458 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 02:48:54,458 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,458 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:54,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 02:48:54,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:54,460 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:54,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 02:48:54,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:54,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:54,461 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:54,461 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 02:48:54,461 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:54,461 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:54,461 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 02:48:54,461 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:54,461 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,461 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:54,462 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:48:54,462 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:48:54,462 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards] 02:48:54,462 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,462 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,462 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:48:54,462 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 02:48:54,462 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 02:48:54,462 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 02:48:54,463 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 02:48:54,463 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 02:48:54,464 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 02:48:54,464 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 02:48:54,464 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:54,464 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,464 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 02:48:54,464 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 02:48:54,464 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 02:48:54,465 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 02:48:54,465 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,466 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,466 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 02:48:54,466 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 02:48:54,466 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 02:48:54,466 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 02:48:54,466 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,477 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 02:48:54,477 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,478 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 02:48:54,478 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,478 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,480 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:54,480 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 02:48:54,480 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:54,480 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:54,480 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 02:48:54,480 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:54,480 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,480 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:54,481 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:54,481 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:48:54,481 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:48:54,481 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 02:48:54,481 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:48:54,481 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:48:54,481 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 02:48:54,481 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:48:54,481 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards] 02:48:54,481 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,481 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:48:54,481 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,481 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,481 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:48:54,481 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 02:48:54,482 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 02:48:54,482 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 02:48:54,482 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 02:48:54,482 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 02:48:54,482 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 02:48:54,482 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 02:48:54,482 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:54,482 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 02:48:54,482 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:54,483 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,483 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 02:48:54,483 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 02:48:54,483 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 02:48:54,483 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 02:48:54,483 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#65172221] 02:48:54,483 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:54,483 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1608293555] 02:48:54,483 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1608293555] 02:48:54,483 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,483 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 02:48:54,484 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 02:48:54,484 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 02:48:54,484 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1721755042] 02:48:54,484 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:54,484 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#1313655574] 02:48:54,485 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#1313655574] 02:48:54,485 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1666689734] 02:48:54,485 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:54,485 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1217227812] 02:48:54,485 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1217227812] 02:48:54,485 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 02:48:54,485 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 02:48:54,487 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:48:54,488 PM [main] [INFO] ClusterAdminRpcService - Change member voting states for all shards 02:48:54,488 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testChangeMemberVotingStatesForAllShards 02:48:54,489 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testChangeMemberVotingStatesForAllShards 02:48:54,489 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 02:48:54,489 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 02:48:54,490 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 02:48:54,490 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 02:48:54,490 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 02:48:54,490 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 02:48:54,491 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,491 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,491 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 02:48:54,492 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,492 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards)] 02:48:54,492 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 02:48:54,492 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,492 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards)] 02:48:54,494 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 02:48:54,494 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:48:54,494 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 02:48:54,494 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 02:48:54,494 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:48:54,494 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 02:48:54,494 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]] 02:48:54,494 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]] 02:48:54,494 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,494 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,494 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,494 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,494 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=d40fe006-3188-4a71-8123-9aa95dfe380d}, logIndex: 0, isSendImmediate: true 02:48:54,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 02:48:54,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 02:48:54,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,494 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:54,494 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=7f73f70e-134b-491b-b612-3e76c38f5cd9}, logIndex: 0, isSendImmediate: true 02:48:54,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,495 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 02:48:54,495 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 02:48:54,495 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,495 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,495 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 02:48:54,495 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,495 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,495 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,495 PM [cluster-test-shard-dispatcher-17] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:54,495 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,496 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 02:48:54,496 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 02:48:54,496 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:54,496 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 02:48:54,496 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,496 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,496 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 02:48:54,497 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,497 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 02:48:54,497 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:54,497 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,497 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,497 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 02:48:54,497 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 02:48:54,497 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 02:48:54,497 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:54,498 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:54,498 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 02:48:54,498 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 02:48:54,498 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,498 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,498 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 02:48:54,498 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,499 PM [cluster-test-akka.actor.default-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards} from local shard akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 02:48:54,499 PM [cluster-test-akka.actor.default-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,499 PM [cluster-test-akka.actor.default-dispatcher-35] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 02:48:54,499 PM [cluster-test-akka.actor.default-dispatcher-35] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,499 PM [cluster-test-akka.actor.default-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards} from local shard akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,499 PM [cluster-test-akka.actor.default-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 02:48:54,499 PM [cluster-test-akka.actor.default-dispatcher-35] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 02:48:54,499 PM [cluster-test-akka.actor.default-dispatcher-35] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 02:48:54,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,500 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 02:48:54,500 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,500 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 02:48:54,500 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,500 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 02:48:54,500 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 02:48:54,500 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 02:48:54,500 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 02:48:54,500 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,501 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 02:48:54,501 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 02:48:54,501 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,501 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 02:48:54,501 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,501 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards)] 02:48:54,501 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 02:48:54,501 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,501 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 02:48:54,501 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,502 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,502 PM [cluster-test-shard-dispatcher-36] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 02:48:54,502 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,502 PM [cluster-test-shard-dispatcher-36] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards)] 02:48:54,502 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 02:48:54,502 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:48:54,502 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]] 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=8b63895f-99f1-4068-ae9c-01919be6f592}, logIndex: 0, isSendImmediate: true 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:48:54,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]] 02:48:54,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=5e8baab8-f755-43c8-9f63-3bdaf84e0ec3}, logIndex: 0, isSendImmediate: true 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 02:48:54,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 02:48:54,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 02:48:54,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,503 PM [cluster-test-shard-dispatcher-17] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:54,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,504 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 02:48:54,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 02:48:54,504 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:54,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 02:48:54,505 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,505 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 02:48:54,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:54,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 02:48:54,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 02:48:54,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 02:48:54,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:54,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,506 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,506 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:54,507 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 02:48:54,507 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 02:48:54,507 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,507 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 02:48:54,507 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 02:48:54,507 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 02:48:54,507 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:54,507 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:54,507 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 02:48:54,507 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 02:48:54,507 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,507 PM [cluster-test-akka.actor.default-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards} from local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,507 PM [cluster-test-akka.actor.default-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 02:48:54,507 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards} from local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,507 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 02:48:54,507 PM [cluster-test-akka.actor.default-dispatcher-35] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 02:48:54,507 PM [cluster-test-akka.actor.default-dispatcher-35] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 02:48:54,507 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 02:48:54,507 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 02:48:54,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 02:48:54,507 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 02:48:54,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,507 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 02:48:54,508 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:54,508 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#65172221] 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 02:48:54,508 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,508 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,508 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 02:48:54,508 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,508 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 02:48:54,508 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 02:48:54,509 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,509 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:54,509 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1608293555] 02:48:54,509 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 02:48:54,509 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 02:48:54,509 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 02:48:54,509 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1608293555] 02:48:54,509 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#65172221] 02:48:54,510 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 02:48:54,510 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1994637437] 02:48:54,510 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1994637437] 02:48:54,510 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1115559355] 02:48:54,510 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 02:48:54,510 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1432667645] 02:48:54,510 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 02:48:54,510 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1432667645] 02:48:54,510 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:54,510 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1115559355] 02:48:54,510 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 02:48:54,510 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#1651316941] 02:48:54,510 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#1651316941] 02:48:54,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1721755042] 02:48:54,511 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:54,511 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#1313655574] 02:48:54,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#1313655574] 02:48:54,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1721755042] 02:48:54,511 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 02:48:54,511 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#1354254925] 02:48:54,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#1354254925] 02:48:54,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#509117034] 02:48:54,511 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 02:48:54,511 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1888490411] 02:48:54,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1888490411] 02:48:54,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#509117034] 02:48:54,511 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 02:48:54,511 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1773877848] 02:48:54,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1773877848] 02:48:54,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1666689734] 02:48:54,511 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:54,512 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1217227812] 02:48:54,512 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1217227812] 02:48:54,512 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1666689734] 02:48:54,512 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 02:48:54,512 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#200891389] 02:48:54,512 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#200891389] 02:48:54,512 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1498472954] 02:48:54,512 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 02:48:54,512 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1459473491] 02:48:54,512 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1459473491] 02:48:54,512 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1498472954] 02:48:54,512 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 02:48:54,512 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1060178595] 02:48:54,512 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1060178595] 02:48:54,512 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:54,513 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,513 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,513 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 02:48:54,513 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 02:48:54,513 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,513 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 02:48:54,513 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 02:48:54,513 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,513 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 02:48:54,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 02:48:54,514 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 02:48:54,514 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:54,514 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,514 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,514 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 02:48:54,514 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 02:48:54,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 02:48:54,515 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,515 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 02:48:54,515 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,515 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 02:48:54,515 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 02:48:54,515 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 02:48:54,516 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:54,516 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:54,521 PM [cluster-test-akka.actor.default-dispatcher-34] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:54,521 PM [cluster-test-akka.actor.default-dispatcher-34] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:54,521 PM [cluster-test-akka.actor.default-dispatcher-34] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:54,521 PM [cluster-test-akka.actor.default-dispatcher-34] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:54,522 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:48:54,533 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:54,533 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,533 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,533 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 02:48:54,534 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 02:48:54,534 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 02:48:54,534 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:54,534 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:54,534 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 02:48:54,534 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 02:48:54,535 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 02:48:54,536 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 02:48:54,536 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:54,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,536 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 02:48:54,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 02:48:54,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 02:48:54,536 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:54,536 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:54,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 02:48:54,536 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 02:48:54,536 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 02:48:54,536 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 02:48:54,537 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:54,537 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:55,249 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 02:48:55,528 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-9082138879440525737] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:48:55,557 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:55,559 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [6857785753816278512] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:48:55,559 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:55,561 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:48:55.563] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:55,575 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:55,576 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:55,576 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:55,576 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 02:48:55,576 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 02:48:55,576 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 02:48:55,576 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 02:48:55,576 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 02:48:55,576 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5779fb8e 02:48:55,576 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e1eadf1 02:48:55,576 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5779fb8e 02:48:55,576 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e1eadf1 02:48:55,576 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 02:48:55,576 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 02:48:55,576 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 02:48:55,576 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 02:48:55,576 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:55,576 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:55,577 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:55,577 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:55,577 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 02:48:55,577 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 02:48:56,17 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 02:48:56,17 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 441.3 ms 02:48:56,18 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 02:48:56,18 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 02:48:56,18 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 02:48:56,18 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 02:48:56,18 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 02:48:56,58 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 02:48:56,58 PM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 481.9 ms 02:48:56,58 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 02:48:56,58 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 02:48:56,58 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 02:48:56,59 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 02:48:56,59 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 02:48:56,59 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:48:56,59 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:48:56,60 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 02:48:56,60 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:56,60 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:56,60 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 02:48:56,60 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 02:48:56,61 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 02:48:56,61 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@26a52425 02:48:56,61 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@26a52425 02:48:56,61 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 02:48:56,61 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 02:48:56,61 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 02:48:56,61 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:56,61 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 02:48:56,61 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 02:48:56,61 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:56,61 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 02:48:56,61 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@312e229b 02:48:56,61 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@312e229b 02:48:56,61 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 02:48:56,61 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 02:48:56,61 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:56,61 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:56,397 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 02:48:56,397 PM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 336.5 ms 02:48:56,398 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 02:48:56,398 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 02:48:56,398 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 02:48:56,398 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 02:48:56,398 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 02:48:56,597 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 02:48:56,597 PM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 536.8 ms 02:48:56,597 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 02:48:56,597 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 02:48:56,597 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 02:48:56,598 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 02:48:56,598 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 02:48:56,598 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:48:56,598 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:48:56,599 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:56,600 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:56,603 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:48:56.605] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:56,628 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:48:56,636 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7276817012014688359] 02:48:56,636 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:48:56,639 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:48:56,639 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:48:56,639 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:56,639 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:56,650 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,651 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,657 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testGetPrefixShardRole 02:48:56,657 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:48:56,657 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:48:56,658 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:48:56,662 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1114204571] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,662 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:48:56,663 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:48:56,663 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-default-config_testGetPrefixShardRole 02:48:56,663 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:56,663 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 02:48:56,666 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 02:48:56,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@66779e46 02:48:56,667 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} started 02:48:56,668 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,669 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:56,669 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config_testGetPrefixShardRole 02:48:56,669 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,670 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 02:48:56,670 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole to akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 02:48:56,670 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-default-config_testGetPrefixShardRole on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole on shard member-1-shard-default-config_testGetPrefixShardRole whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-1628406485] 02:48:56,670 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-default-config_testGetPrefixShardRole, persistent : true 02:48:56,670 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-1628406485] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,671 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-default-config_testGetPrefixShardRole with journal batch size 1 02:48:56,671 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testGetPrefixShardRole 02:48:56,671 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/shard-member-1:snapshot-read#-452025949] 02:48:56,671 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-528930264 created and ready for shard:member-1-shard-default-config_testGetPrefixShardRole 02:48:56,671 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-528930264] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,672 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#1592455997] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,672 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/shard-member-1:snapshot-read#-452025949] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,673 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:48:56,674 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:56,674 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:56,674 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [default] 02:48:56,674 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-config_testGetPrefixShardRole is initialized 02:48:56,674 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 02:48:56,674 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-config_testGetPrefixShardRole is no longer the leader 02:48:56,674 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Follower): Received TimeoutNow - switching to Candidate 02:48:56,674 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Election: Candidate has following voting peers: [] 02:48:56,674 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: Set currentTerm=1, votedFor=member-1-shard-default-config_testGetPrefixShardRole 02:48:56,674 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Starting new election term 1 02:48:56,674 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:56,674 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 02:48:56,674 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-config_testGetPrefixShardRole is no longer the leader 02:48:56,674 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from null to Follower 02:48:56,674 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Follower to Candidate 02:48:56,677 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Received ElectionTimeout 02:48:56,677 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Leader): Election: Leader has following peers: [] 02:48:56,677 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:48:56,677 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 02:48:56,677 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: became leader with frontend state for [] 02:48:56,677 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 02:48:56,677 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:48:56,677 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Candidate to Leader 02:48:56,677 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , registered listener akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole 02:48:56,677 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-default-oper_testGetPrefixShardRole 02:48:56,677 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config_testGetPrefixShardRole, leaderId=member-1-shard-default-config_testGetPrefixShardRole, leaderPayloadVersion=13] 02:48:56,677 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-default-config_testGetPrefixShardRole from Candidate to Leader 02:48:56,677 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testGetPrefixShardRole is ready 02:48:56,677 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:56,678 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 02:48:56,678 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} started 02:48:56,682 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7352581a 02:48:56,682 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:56,682 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-oper_testGetPrefixShardRole 02:48:56,682 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 02:48:56,682 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-oper_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole to akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 02:48:56,682 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-default-oper_testGetPrefixShardRole on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole on shard member-1-shard-default-oper_testGetPrefixShardRole whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-208735628] 02:48:56,683 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-default-oper_testGetPrefixShardRole, persistent : true 02:48:56,683 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-208735628] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,683 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/shard-member-1:snapshot-read#-192464853] 02:48:56,683 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/shard-member-1:snapshot-read#-192464853] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,684 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#-1702303662] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,684 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#-1702303662 created and ready for shard:member-1-shard-default-oper_testGetPrefixShardRole 02:48:56,684 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-default-oper_testGetPrefixShardRole with journal batch size 1 02:48:56,685 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:56,685 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:48:56,685 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [default] 02:48:56,685 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-oper_testGetPrefixShardRole is initialized 02:48:56,685 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 02:48:56,685 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-oper_testGetPrefixShardRole is no longer the leader 02:48:56,685 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Follower): Received TimeoutNow - switching to Candidate 02:48:56,685 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Election: Candidate has following voting peers: [] 02:48:56,685 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Set currentTerm=1, votedFor=member-1-shard-default-oper_testGetPrefixShardRole 02:48:56,685 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Starting new election term 1 02:48:56,685 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:48:56,685 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 02:48:56,685 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-oper_testGetPrefixShardRole is no longer the leader 02:48:56,686 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from null to Follower 02:48:56,686 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Follower to Candidate 02:48:56,686 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Received ElectionTimeout 02:48:56,686 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Leader): Election: Leader has following peers: [] 02:48:56,686 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:48:56,686 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 02:48:56,686 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: became leader with frontend state for [] 02:48:56,686 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Candidate to Leader 02:48:56,686 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , registered listener akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole 02:48:56,686 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-oper_testGetPrefixShardRole, leaderId=member-1-shard-default-oper_testGetPrefixShardRole, leaderPayloadVersion=13] 02:48:56,687 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-default-oper_testGetPrefixShardRole from Candidate to Leader 02:48:56,687 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testGetPrefixShardRole is ready 02:48:56,729 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1114204571] 02:48:56,729 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : default 02:48:56,729 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-1628406485] 02:48:56,729 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 02:48:56,730 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-default-config_testGetPrefixShardRole 02:48:56,730 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 02:48:56,730 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onShutDown 02:48:56,730 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@bdbcd5a 02:48:56,730 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-default-config_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@bdbcd5a 02:48:56,730 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-default-config_testGetPrefixShardRole 02:48:56,730 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: Aborting 0 pending queued transactions 02:48:56,731 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:48:56,731 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:48:56,731 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 02:48:56,731 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-default-oper_testGetPrefixShardRole 02:48:56,731 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1114204571] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1114204571] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,731 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 02:48:56,731 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onShutDown 02:48:56,731 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@4dbba7b9 02:48:56,731 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-default-oper_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@4dbba7b9 02:48:56,732 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-default-oper_testGetPrefixShardRole 02:48:56,732 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Aborting 0 pending queued transactions 02:48:56,732 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:48:56,732 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:48:56,732 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007ff1087ccc30] to Actor[akka://cluster-test/user/$a#1606720007] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1606720007] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,736 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:56,758 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:48:56,769 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5108571155090073050] 02:48:56,769 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:48:56,769 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:48:56,769 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:48:56,770 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:56,770 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:56,778 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:48:56,778 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:48:56,779 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,779 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:48:56,780 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,783 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,788 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:48:56,793 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#512484146] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,808 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:48:56,808 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,809 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,810 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:56,810 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,810 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,811 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 02:48:56,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1c261da3 02:48:56,812 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 02:48:56,813 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:56,813 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,813 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,814 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,814 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,814 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#843966485] 02:48:56,814 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,814 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,814 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,814 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1198941233] 02:48:56,814 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,815 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 02:48:56,816 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 02:48:56,816 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,817 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#1142513512] 02:48:56,817 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1198941233] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,817 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#1142513512] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,818 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#329732139] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,818 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#329732139 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,821 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-2092545112] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,821 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 02:48:56,821 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#843966485] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,822 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#169767284] 02:48:56,822 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 02:48:56,822 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:48:56,822 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-85785663 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,822 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#169767284] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,822 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-85785663] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,823 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:56,823 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 02:48:56,823 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:56,823 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 02:48:56,823 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 02:48:56,823 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:56,823 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:56,823 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:56,823 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:56,823 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 02:48:56,823 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 02:48:56,823 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 02:48:56,824 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,824 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 02:48:56,825 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 02:48:56,826 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 02:48:56,828 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:48:56,828 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,828 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:56,828 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 02:48:56,828 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:56,828 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 02:48:56,828 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 02:48:56,828 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:56,828 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:56,828 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 02:48:56,828 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 02:48:56,828 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:56,828 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:56,829 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 02:48:56,829 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,829 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:56,829 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,829 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,829 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6662aaf5 02:48:56,829 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:56,829 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,829 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,829 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 02:48:56,830 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#21837243] 02:48:56,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,830 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-2095256223] 02:48:56,830 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 02:48:56,831 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 02:48:56,831 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-2095256223] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,831 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#-1320483011] 02:48:56,831 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1059875926 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,831 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1059875926] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,832 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 02:48:56,832 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1732119077 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:56,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 02:48:56,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:56,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 02:48:56,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 02:48:56,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:56,832 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:56,833 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 02:48:56,833 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 02:48:56,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:56,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:56,833 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 02:48:56,833 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,833 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 02:48:56,833 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#2044744146] 02:48:56,833 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 02:48:56,834 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:56,834 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 02:48:56,834 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:56,834 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 02:48:56,834 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 02:48:56,834 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:56,834 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:56,834 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 02:48:56,834 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 02:48:56,834 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:56,834 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:56,834 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 02:48:56,834 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,834 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 02:48:56,840 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:48:56,846 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5889475192569206073] 02:48:56,846 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:48:56,849 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:56,849 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:48:56,858 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:56,858 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:56,860 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,861 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,863 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,873 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1433867535] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,880 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 02:48:56,881 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 02:48:56,881 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,882 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,886 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 02:48:56,888 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7f73dd53 02:48:56,888 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 02:48:56,889 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,889 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:56,889 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,890 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,890 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 02:48:56,891 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,891 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 02:48:56,891 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 02:48:56,892 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 02:48:56,894 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,895 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 02:48:56,896 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-642261696] 02:48:56,896 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1575599501 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,896 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#663735314] 02:48:56,896 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1262347422 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,896 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#178675854] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,896 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-110094311] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,896 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1999130632] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,896 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1575599501] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,896 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-642261696] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,896 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1262347422] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,896 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#663735314] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,898 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:56,898 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 02:48:56,899 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 02:48:56,899 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,899 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,899 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:56,899 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 02:48:56,899 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 02:48:56,899 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:56,899 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:56,901 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 02:48:56,905 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 02:48:56,905 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 02:48:56,905 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5fd2d964 02:48:56,905 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:56,905 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:56,905 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 02:48:56,905 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,905 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 02:48:56,906 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 02:48:56,907 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:56,907 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 02:48:56,907 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:56,907 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 02:48:56,907 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 02:48:56,907 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:56,907 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:56,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 02:48:56,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 02:48:56,907 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:56,907 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:56,907 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 02:48:56,908 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,908 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 02:48:56,910 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:56,910 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,910 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,911 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 02:48:56,912 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 02:48:56,912 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 02:48:56,912 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1418489886 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,912 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#1795901666] 02:48:56,913 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1359185174] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,913 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1144848555 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,913 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-1697600167] 02:48:56,913 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 02:48:56,913 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#427300777] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,914 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:56,914 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 02:48:56,914 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:56,914 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 02:48:56,914 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:56,915 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:56,915 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 02:48:56,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:56,915 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 02:48:56,915 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 02:48:56,915 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,915 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,915 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 02:48:56,915 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 02:48:56,917 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:48:56,921 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:48:56,921 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:56,921 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,921 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,921 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,921 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,921 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,921 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,921 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:56,921 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,921 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,921 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,921 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,921 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,921 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,925 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:48:56,931 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4025042682827316678] 02:48:56,932 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 02:48:56,934 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:56,934 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 02:48:56,937 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:56,937 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:56,947 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,950 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,953 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,971 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-137206973] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,975 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 02:48:56,981 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:48:56,982 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 02:48:56,985 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 02:48:56,985 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,986 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,986 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:56,986 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,986 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,988 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 02:48:56,989 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@43c15b5b 02:48:56,990 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 02:48:56,990 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:56,990 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,991 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:56,991 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,991 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,991 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#839232235] 02:48:56,991 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,991 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,991 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,991 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1377570004] 02:48:56,992 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 02:48:56,999 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1377570004] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:56,999 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 02:48:56,999 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#296227897 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:56,999 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#1693149628] 02:48:57,0 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 02:48:57,0 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#839232235] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:57,0 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#296227897] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:57,0 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#1693149628] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:57,0 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1922813118] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:57,0 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#1244069612] 02:48:57,0 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#1244069612] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:57,1 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:57,1 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 02:48:57,3 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,4 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1166384378] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:57,4 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 02:48:57,5 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1922813118 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,5 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:57,5 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 02:48:57,5 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:57,5 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 02:48:57,5 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 02:48:57,5 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:57,5 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:57,5 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 02:48:57,5 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:57,5 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:57,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 02:48:57,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 02:48:57,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:57,6 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:57,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:57,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:57,6 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 02:48:57,6 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 02:48:57,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 02:48:57,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 02:48:57,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 02:48:57,8 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 02:48:57,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:57,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:57,13 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,13 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,13 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 02:48:57,13 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,14 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 02:48:57,14 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 02:48:57,14 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 02:48:57,14 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,14 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 02:48:57,15 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,15 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:57,16 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,16 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@b4bd4cb 02:48:57,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:57,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,17 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,17 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,17 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1591172044] 02:48:57,17 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,17 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,17 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,17 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1930408918] 02:48:57,17 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1591172044] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:57,17 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 02:48:57,18 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1930408918] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:57,18 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 02:48:57,18 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#942632489 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,18 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#-2061501830] 02:48:57,19 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1831236441 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,19 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#1523134474] 02:48:57,20 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 02:48:57,20 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 02:48:57,21 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:57,21 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 02:48:57,21 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:57,22 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 02:48:57,22 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 02:48:57,22 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:57,22 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:57,22 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 02:48:57,22 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 02:48:57,22 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:57,22 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:57,22 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 02:48:57,21 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:57,22 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 02:48:57,22 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:57,22 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 02:48:57,22 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 02:48:57,22 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:57,22 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:57,22 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 02:48:57,22 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 02:48:57,22 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:57,22 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:57,22 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 02:48:57,22 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,22 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,23 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,23 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 02:48:57,23 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,23 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 02:48:57,786 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 02:48:57,786 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:48:57,786 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:57,786 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,787 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,794 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,816 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#512484146] 02:48:57,816 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:57,816 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#843966485] 02:48:57,816 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#843966485] 02:48:57,817 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#512484146] 02:48:57,817 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:57,817 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#843966485] 02:48:57,817 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#843966485] 02:48:57,818 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 02:48:57,818 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,818 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,818 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 02:48:57,818 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 02:48:57,818 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 02:48:57,818 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 02:48:57,818 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 02:48:57,818 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:57,818 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 02:48:57,818 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 02:48:57,819 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,819 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,820 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]], state: Idle 02:48:57,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]], state: Idle 02:48:57,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:48:57,820 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:48:57,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 02:48:57,820 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 02:48:57,820 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,820 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]], state: Idle 02:48:57,820 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 02:48:57,820 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:48:57,820 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 02:48:57,820 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 02:48:57,820 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]], state: Idle 02:48:57,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 02:48:57,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:48:57,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 02:48:57,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 02:48:57,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 02:48:57,820 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 02:48:57,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 02:48:57,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 02:48:57,820 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 02:48:57,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 02:48:57,820 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,820 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,821 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 02:48:57,821 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 02:48:57,821 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 02:48:57,821 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 02:48:57,821 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,821 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 02:48:57,821 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,821 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,821 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,821 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,821 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,821 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,821 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 02:48:57,821 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 02:48:57,821 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 02:48:57,821 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:57,821 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:57,821 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:57,821 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:57,821 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:57,821 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:57,821 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 02:48:57,821 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 02:48:57,821 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 02:48:57,821 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 02:48:57,821 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 02:48:57,821 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 02:48:57,821 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 02:48:57,821 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,821 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,821 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 02:48:57,821 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 02:48:57,821 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 02:48:57,821 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 02:48:57,822 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 02:48:57,823 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:57,823 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 02:48:57,823 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:48:57,823 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:57,823 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:48:57,823 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 02:48:57,823 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:57,823 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 02:48:57,823 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:48:57,823 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 02:48:57,823 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:48:57,823 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,823 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 02:48:57,823 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 02:48:57,823 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 02:48:57,823 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,823 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 02:48:57,823 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:57,823 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 02:48:57,823 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:48:57,823 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:48:57,823 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 02:48:57,823 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 02:48:57,823 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:48:57,823 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:48:57,823 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 02:48:57,823 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 02:48:57,824 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 02:48:57,824 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,824 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,824 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 02:48:57,824 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 02:48:57,824 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:57,824 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 02:48:57,824 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:48:57,824 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:57,824 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:48:57,824 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 02:48:57,824 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 02:48:57,824 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:48:57,824 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:48:57,824 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 02:48:57,824 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 02:48:57,824 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,824 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 02:48:57,824 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 02:48:57,825 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,825 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 02:48:57,825 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:57,825 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 02:48:57,825 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:48:57,825 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:48:57,825 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:48:57,825 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 02:48:57,825 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 02:48:57,825 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:48:57,825 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:48:57,825 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 02:48:57,825 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 02:48:57,825 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,825 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 02:48:57,825 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 02:48:57,825 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,825 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 02:48:57,825 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 02:48:57,825 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 02:48:57,825 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 02:48:57,825 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 02:48:57,825 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:57,825 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:57,825 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 02:48:57,825 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 02:48:57,825 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 02:48:57,825 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 02:48:57,825 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 02:48:57,825 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 02:48:57,825 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader elected 02:48:57,825 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 02:48:57,825 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:57,826 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,826 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 02:48:57,826 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=ee7bfe1c-db54-4d08-ba5b-d42a18a15485}, logIndex: 1, isSendImmediate: true 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader elected 02:48:57,826 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 02:48:57,826 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader elected 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 02:48:57,826 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=b44d0333-6a43-49d2-a1f8-e1b70670a60c}, logIndex: 1, isSendImmediate: true 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:57,826 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=bc069bf0-74a1-43d5-8dbb-07f1bbe8e938}, logIndex: 1, isSendImmediate: true 02:48:57,826 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:57,826 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] 02:48:57,826 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,826 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,826 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 02:48:57,826 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,827 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 02:48:57,826 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 02:48:57,827 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 02:48:57,827 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 02:48:57,827 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,827 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader elected 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 02:48:57,827 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=da8a518b-1914-43a0-a192-d5f65013bd1e}, logIndex: 1, isSendImmediate: true 02:48:57,827 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 02:48:57,827 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 02:48:57,827 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:57,827 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 02:48:57,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:48:57,827 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,828 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] 02:48:57,828 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,828 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,828 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,828 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 02:48:57,828 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 02:48:57,828 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 02:48:57,828 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,828 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,828 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 02:48:57,828 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,828 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 02:48:57,828 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#512484146] 02:48:57,828 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,828 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:57,828 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#843966485] 02:48:57,828 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 02:48:57,828 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 02:48:57,829 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 02:48:57,829 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 02:48:57,829 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#843966485] 02:48:57,829 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,829 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 02:48:57,829 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,829 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 02:48:57,829 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,829 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 02:48:57,829 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 02:48:57,829 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 02:48:57,829 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 02:48:57,829 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 02:48:57,829 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 02:48:57,829 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 02:48:57,829 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,829 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 02:48:57,830 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 02:48:57,830 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#512484146] 02:48:57,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 02:48:57,830 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 02:48:57,830 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1198941233] 02:48:57,830 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1198941233] 02:48:57,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 02:48:57,830 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 02:48:57,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,830 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,830 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,830 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,830 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,830 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,830 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,830 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,830 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,831 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-2092545112] 02:48:57,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 02:48:57,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 02:48:57,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,831 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,832 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,832 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,832 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 02:48:57,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,832 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,832 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#21837243] 02:48:57,831 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,832 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,832 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 02:48:57,832 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,832 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,832 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,832 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,832 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,832 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,832 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,832 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,832 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,833 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#21837243] 02:48:57,833 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-2092545112] 02:48:57,833 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 02:48:57,833 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,833 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 02:48:57,833 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:57,833 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,833 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,833 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,833 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 02:48:57,833 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:57,833 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,833 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,833 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 02:48:57,833 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,833 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,833 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 02:48:57,833 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 02:48:57,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,833 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:57,833 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 02:48:57,833 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:57,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,833 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 02:48:57,833 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,833 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:57,833 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-2095256223] 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 02:48:57,834 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-2095256223] 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:57,834 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 02:48:57,834 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,834 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 02:48:57,834 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 02:48:57,834 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 02:48:57,834 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,834 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,834 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 02:48:57,834 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:57,834 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 02:48:57,834 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,834 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,834 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,834 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 02:48:57,834 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:57,834 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 02:48:57,834 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:57,834 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,834 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,834 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,835 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 02:48:57,835 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:57,835 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 02:48:57,835 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,835 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,835 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,835 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,835 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 02:48:57,835 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:57,835 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 02:48:57,835 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,835 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,835 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 02:48:57,835 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:57,835 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,835 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,835 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 02:48:57,835 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 02:48:57,835 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,835 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,835 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,835 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,836 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1433867535] 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:57,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]] - lastApplied: 2, lastIndex: 2 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]] - lastApplied: 2, lastIndex: 2 02:48:57,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 02:48:57,836 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:57,836 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]] - lastApplied: 2, lastIndex: 2 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,836 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,836 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,837 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,837 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,837 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 02:48:57,837 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 02:48:57,837 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 02:48:57,837 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 02:48:57,837 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 02:48:57,837 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 02:48:57,837 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 02:48:57,837 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 02:48:57,837 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 02:48:57,837 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 02:48:57,837 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 02:48:57,837 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#178675854] 02:48:57,837 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#178675854] 02:48:57,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 02:48:57,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 02:48:57,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 02:48:57,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 02:48:57,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 02:48:57,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 02:48:57,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 02:48:57,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,838 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 02:48:57,838 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1433867535] 02:48:57,838 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 02:48:57,838 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 02:48:57,838 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 02:48:57,838 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,838 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 02:48:57,839 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,839 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 02:48:57,839 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 02:48:57,839 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-110094311] 02:48:57,839 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-110094311] 02:48:57,838 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 02:48:57,839 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1999130632] 02:48:57,839 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 02:48:57,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 02:48:57,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 02:48:57,839 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,839 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 02:48:57,839 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 02:48:57,839 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 02:48:57,839 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 02:48:57,839 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,839 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,839 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1359185174] 02:48:57,839 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 02:48:57,839 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 02:48:57,839 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,839 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,839 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1359185174] 02:48:57,839 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 02:48:57,839 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 02:48:57,839 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,839 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 02:48:57,840 PM [cluster-test-shard-dispatcher-33] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 02:48:57,840 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]] - lastApplied: 2, lastIndex: 2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] is present: existingEntryTerm: 2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] is present: existingEntryTerm: 2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 2, entries to be added from: 2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,840 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 02:48:57,840 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1999130632] 02:48:57,840 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 02:48:57,840 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,840 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,840 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] is present: existingEntryTerm: 2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] is present: existingEntryTerm: 2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 2, entries to be added from: 2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:57,840 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 02:48:57,840 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 02:48:57,840 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 02:48:57,840 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 02:48:57,841 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 02:48:57,841 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 02:48:57,840 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 02:48:57,840 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,841 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,841 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 02:48:57,841 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 02:48:57,841 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,841 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 02:48:57,841 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#427300777] 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] is present: existingEntryTerm: 2 02:48:57,841 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] is present: existingEntryTerm: 2 02:48:57,841 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 2, entries to be added from: 2 02:48:57,841 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 02:48:57,841 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#427300777] 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 02:48:57,841 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,841 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:57,841 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 02:48:57,841 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 02:48:57,841 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 02:48:57,841 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 02:48:57,841 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,841 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 02:48:57,841 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 02:48:57,841 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 02:48:57,841 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,841 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,841 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 02:48:57,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 02:48:57,840 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,841 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,841 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 02:48:57,841 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 02:48:57,842 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 02:48:57,842 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 02:48:57,842 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 02:48:57,842 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 02:48:57,842 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 02:48:57,842 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:48:57,842 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-137206973] 02:48:57,842 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 02:48:57,842 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:48:57,842 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,842 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#839232235] 02:48:57,842 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#839232235] 02:48:57,843 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-137206973] 02:48:57,843 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 02:48:57,843 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1377570004] 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:48:57,843 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1377570004] 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] is present: existingEntryTerm: 2 02:48:57,843 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1166384378] 02:48:57,843 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] is present: existingEntryTerm: 2 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 2, entries to be added from: 2 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:48:57,843 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1591172044] 02:48:57,843 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1591172044] 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 02:48:57,843 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1166384378] 02:48:57,843 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 02:48:57,843 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1930408918] 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 02:48:57,843 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1930408918] 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 02:48:57,843 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:48:57,844 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#512484146] 02:48:57,844 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:48:57,844 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 02:48:57,844 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#843966485] 02:48:57,844 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#843966485] 02:48:57,845 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-2092545112] 02:48:57,845 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 02:48:57,845 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#21837243] 02:48:57,845 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#21837243] 02:48:57,845 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 02:48:57,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,845 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,845 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 02:48:57,845 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 02:48:57,845 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 02:48:57,846 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,846 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,846 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 02:48:57,846 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 02:48:57,846 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 02:48:57,846 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 02:48:57,846 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 02:48:57,846 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,846 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,846 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 02:48:57,846 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 02:48:57,847 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 02:48:57,847 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,847 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,847 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 02:48:57,847 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 02:48:57,847 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 02:48:57,847 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 02:48:57,847 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:57,847 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:57,850 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:57,850 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:57,851 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:48:57,868 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 02:48:57,869 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,869 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,869 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 02:48:57,869 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 02:48:57,869 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 02:48:57,870 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,870 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 02:48:57,870 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,870 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 02:48:57,871 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 02:48:57,871 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 02:48:57,873 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 02:48:57,873 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,873 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,873 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 02:48:57,873 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 02:48:57,873 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 02:48:57,873 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,873 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,873 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 02:48:57,873 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 02:48:57,874 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 02:48:57,874 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 02:48:57,875 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:48:57,875 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:48:57,875 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:57,875 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:57,879 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:57,879 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:48:57,880 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:48:57.881] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:57,890 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 02:48:57,891 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,891 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:57,891 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 02:48:57,891 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 02:48:57,891 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 02:48:57,891 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 02:48:57,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 02:48:57,892 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@15553819 02:48:57,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 02:48:57,892 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@15553819 02:48:57,892 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 02:48:57,892 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 02:48:57,892 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,892 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,892 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 02:48:57,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2a3b82d6 02:48:57,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2a3b82d6 02:48:57,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 02:48:57,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 02:48:57,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:57,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:58,476 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 02:48:58,476 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 02:48:58,476 PM [cluster-test-shard-dispatcher-34] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 584.2 ms 02:48:58,476 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 02:48:58,476 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 02:48:58,476 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 02:48:58,476 PM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 583.9 ms 02:48:58,476 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 02:48:58,476 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 02:48:58,476 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 02:48:58,476 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:58,476 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 02:48:58,477 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 02:48:58,477 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 02:48:58,477 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:48:58,477 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:48:58,477 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 02:48:58,478 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:58,478 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:58,478 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 02:48:58,478 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 02:48:58,478 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 02:48:58,478 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 02:48:58,478 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 02:48:58,478 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 02:48:58,478 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@24fc69e1 02:48:58,479 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@24fc69e1 02:48:58,479 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 02:48:58,479 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 02:48:58,479 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 02:48:58,479 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3adb2348 02:48:58,479 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3adb2348 02:48:58,479 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:58,479 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 02:48:58,479 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 02:48:58,479 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:58,479 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:58,479 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:48:58,796 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 02:48:58,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 02:48:58,796 PM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 317.4 ms 02:48:58,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 02:48:58,796 PM [cluster-test-shard-dispatcher-33] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 317.3 ms 02:48:58,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 02:48:58,796 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 02:48:58,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 02:48:58,796 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 02:48:58,796 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 02:48:58,796 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:58,796 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 02:48:58,796 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 02:48:58,796 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 02:48:58,797 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:48:58,797 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:48:58,798 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:48:58,798 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:48:58,806 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 02:48:59,818 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:48:59.820] [cluster-test-akka.actor.internal-dispatcher-5] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:59,834 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:48:59,842 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5771435623689411196] 02:48:59,842 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:48:59,844 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:48:59,844 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:48:59,846 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:59,846 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:59,856 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:59,858 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:59,860 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:48:59,860 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:48:59,860 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:48:59,860 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,863 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#296550892] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,865 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:48:59,866 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:48:59,866 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,866 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,866 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:59,866 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,866 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,872 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 02:48:59,872 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@22e0a3c0 02:48:59,873 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 02:48:59,873 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:59,876 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:59,877 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:59,877 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,877 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,877 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,877 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#596402182] 02:48:59,877 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,878 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 02:48:59,878 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,878 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,878 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-298411493] 02:48:59,878 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#596402182] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,878 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 02:48:59,878 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-298411493] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,879 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 02:48:59,879 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#906178783 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,879 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#906178783] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,879 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,880 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-2078848018] 02:48:59,880 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-2078848018] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,880 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-378061802] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,881 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:48:59,881 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-376146298 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,881 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-376146298] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,881 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-537320746] 02:48:59,881 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-537320746] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,882 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 02:48:59,882 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:48:59,882 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:59,882 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 02:48:59,882 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:59,882 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,883 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 02:48:59,883 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 02:48:59,883 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,883 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:59,883 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:59,883 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:48:59,883 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,883 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,883 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 02:48:59,883 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 02:48:59,883 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:48:59,883 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:48:59,883 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 02:48:59,883 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,883 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 02:48:59,884 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:59,884 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 02:48:59,884 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:59,885 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 02:48:59,885 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 02:48:59,885 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:59,885 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:59,885 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 02:48:59,885 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 02:48:59,885 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:48:59,885 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:48:59,885 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 02:48:59,885 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,885 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 02:48:59,887 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 02:48:59,887 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7dc46056 02:48:59,887 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 02:48:59,888 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:59,888 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,889 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,889 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,889 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 02:48:59,889 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1623182374] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,889 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-669825860 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,890 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 02:48:59,890 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-669825860] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,890 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-861440821] 02:48:59,889 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1623182374] 02:48:59,892 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,892 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,892 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,892 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#710654609] 02:48:59,892 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 02:48:59,893 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 02:48:59,893 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-2050309403 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,893 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#895371450] 02:48:59,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:59,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 02:48:59,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:59,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 02:48:59,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 02:48:59,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:59,893 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:59,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:48:59,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:48:59,893 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 02:48:59,893 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 02:48:59,893 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 02:48:59,894 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,894 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 02:48:59,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:59,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 02:48:59,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:59,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 02:48:59,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 02:48:59,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:59,894 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:59,894 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 02:48:59,894 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 02:48:59,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:48:59,894 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:48:59,894 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 02:48:59,894 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,894 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 02:48:59,895 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:48:59,902 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8196724518297213337] 02:48:59,902 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:48:59,904 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:48:59,904 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:48:59,911 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:48:59,911 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:48:59,915 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:59,917 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:59,919 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,923 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 02:48:59,927 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1593962586] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,929 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 02:48:59,930 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,931 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,932 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 02:48:59,935 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 02:48:59,935 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:59,941 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:48:59,941 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@af700e1 02:48:59,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:59,954 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,953 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,956 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,958 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-828214800] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,959 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 02:48:59,959 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-181329484 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,959 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-181329484] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,960 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#1213077861] 02:48:59,960 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#1213077861] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,960 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 02:48:59,961 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 02:48:59,961 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1611441101] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,961 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#25124073 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,961 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#1949534078] 02:48:59,961 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#25124073] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,961 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#1949534078] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,961 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 02:48:59,962 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-943847062] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,965 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 02:48:59,966 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:59,967 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 02:48:59,967 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:59,967 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 02:48:59,967 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 02:48:59,967 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:59,967 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:59,967 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:48:59,967 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:48:59,967 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 02:48:59,969 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 02:48:59,969 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 02:48:59,969 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,969 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 02:48:59,970 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:59,970 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 02:48:59,970 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:59,971 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 02:48:59,971 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 02:48:59,971 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:59,971 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:59,971 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 02:48:59,971 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 02:48:59,971 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:48:59,971 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:48:59,971 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 02:48:59,971 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,971 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 02:48:59,971 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 02:48:59,971 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,971 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,972 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 02:48:59,972 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7426845f 02:48:59,973 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 02:48:59,973 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:48:59,973 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,974 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 02:48:59,974 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 02:48:59,976 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#1892712115] 02:48:59,976 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-134984863] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,976 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,977 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-2084933793] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:48:59,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:59,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 02:48:59,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:59,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 02:48:59,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 02:48:59,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:59,977 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:59,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:48:59,977 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:48:59,980 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 02:48:59,980 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-517406710] 02:48:59,980 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1014119934 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,980 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 02:48:59,981 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 02:48:59,981 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 02:48:59,982 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:48:59,982 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 02:48:59,982 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:48:59,982 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 02:48:59,982 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 02:48:59,982 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:48:59,982 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:48:59,982 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 02:48:59,982 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 02:48:59,982 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:48:59,982 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:48:59,982 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 02:48:59,982 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,982 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 02:48:59,991 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-2084933793 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,991 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 02:48:59,991 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:48:59,992 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 02:49:00,29 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:49:00,44 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:49:00,47 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:49:00,48 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:00,48 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,48 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,48 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,48 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,48 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,48 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,48 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:00,48 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,48 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,48 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,48 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,48 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,48 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,53 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7909934857701572934] 02:49:00,54 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 02:49:00,58 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:49:00,58 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 02:49:00,59 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:49:00,59 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:49:00,72 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:00,86 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:00,87 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 02:49:00,89 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:49:00,96 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,101 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1010490443] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:00,101 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 02:49:00,106 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 02:49:00,106 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,106 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,107 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:00,107 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,107 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,109 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 02:49:00,110 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4dee9214 02:49:00,110 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 02:49:00,111 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:00,111 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,111 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:00,112 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:00,115 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,118 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,118 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,118 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-659837958] 02:49:00,118 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,119 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-659837958] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:00,119 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 02:49:00,119 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#2016420441 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,119 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#2016420441] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:00,120 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#693713451] 02:49:00,120 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#693713451] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:00,120 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 02:49:00,121 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:49:00,121 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 02:49:00,121 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:49:00,122 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 02:49:00,122 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 02:49:00,123 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:00,123 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:49:00,123 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:49:00,123 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:49:00,123 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 02:49:00,123 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,123 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#736259823] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:00,123 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 02:49:00,124 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1258174538 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,124 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1258174538] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:00,124 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#1569764538] 02:49:00,124 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#1569764538] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:00,124 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 02:49:00,125 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 02:49:00,125 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:49:00,125 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 02:49:00,125 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:49:00,125 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 02:49:00,125 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 02:49:00,125 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:00,125 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:49:00,125 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:49:00,125 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:49:00,125 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 02:49:00,126 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-279767169] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:00,126 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 02:49:00,126 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 02:49:00,128 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 02:49:00,128 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,128 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,129 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 02:49:00,129 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,129 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,130 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@277fdf21 02:49:00,130 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:00,130 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,130 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,130 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,130 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 02:49:00,131 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 02:49:00,131 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1503592257] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:00,131 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#1168119250] 02:49:00,131 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#615950183 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,131 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:00,131 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,131 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,131 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,131 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,131 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 02:49:00,131 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#736259823] 02:49:00,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 02:49:00,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 02:49:00,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 02:49:00,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 02:49:00,132 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,131 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#1168119250] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:00,132 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,132 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 02:49:00,132 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 02:49:00,132 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-60042080 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,133 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#-1782946635] 02:49:00,133 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 02:49:00,137 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:49:00,137 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 02:49:00,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:49:00,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 02:49:00,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 02:49:00,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:00,138 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:49:00,138 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 02:49:00,138 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 02:49:00,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:49:00,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:49:00,138 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 02:49:00,138 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,138 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,138 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 02:49:00,139 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 02:49:00,139 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 02:49:00,139 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 02:49:00,139 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 02:49:00,139 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 02:49:00,139 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:00,139 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 02:49:00,139 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 02:49:00,139 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 02:49:00,139 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:49:00,139 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:49:00,139 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 02:49:00,139 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,139 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,139 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 02:49:00,231 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:49:00,231 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:49:00,231 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 02:49:00,231 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 02:49:00,232 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 02:49:00,232 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 02:49:00,232 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,232 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,232 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 02:49:00,232 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 02:49:00,232 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 02:49:00,232 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 02:49:00,232 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:49:00,232 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:49:00,232 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:49:00,232 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:49:00,232 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 02:49:00,232 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 02:49:00,233 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 02:49:00,233 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 02:49:00,240 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:49:00,240 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 02:49:00,240 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 02:49:00,240 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,240 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 02:49:00,241 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 02:49:00,241 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:49:00,241 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:49:00,241 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 02:49:00,241 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 02:49:00,311 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:49:00,311 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 02:49:00,311 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 02:49:00,311 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,311 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 02:49:00,311 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 02:49:00,311 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 02:49:00,311 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 02:49:00,311 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 02:49:00,312 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 02:49:00,571 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 02:49:00,571 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,571 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 02:49:00,601 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 02:49:00,601 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,601 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 02:49:00,640 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 02:49:00,641 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,641 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 02:49:00,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 02:49:00,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,651 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 02:49:00,862 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 02:49:00,862 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:49:00,863 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:00,863 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,863 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,863 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,863 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,863 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:00,863 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,863 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,863 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,863 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:00,863 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,863 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,863 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,863 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,863 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,863 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,863 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,864 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,864 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,864 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,864 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:00,864 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,864 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,864 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,864 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,864 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,864 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,864 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,870 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:00,870 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,870 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,870 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,870 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:00,871 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:00,871 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:00,871 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,871 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,938 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:00,938 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,938 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,938 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,938 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,938 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:00,938 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,938 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,938 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,938 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,939 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,939 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,939 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:00,939 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,939 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,939 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,939 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,939 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:00,939 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,939 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,939 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,939 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,939 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,939 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,951 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 02:49:00,951 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,951 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 02:49:00,951 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,951 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,952 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:00,952 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 02:49:00,952 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:49:00,952 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:49:00,952 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 02:49:00,953 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 02:49:00,953 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,953 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:00,953 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:49:00,953 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 02:49:00,953 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:49:00,953 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:49:00,953 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 02:49:00,953 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 02:49:00,953 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,953 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:49:00,954 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 02:49:00,954 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 02:49:00,954 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 02:49:00,954 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 02:49:00,955 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:00,955 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:00,955 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 02:49:00,955 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 02:49:00,955 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 02:49:00,955 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 02:49:00,955 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:00,955 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:00,955 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 02:49:00,955 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 02:49:00,955 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:00,955 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:00,956 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,956 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,956 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 02:49:00,956 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 02:49:00,956 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 02:49:00,956 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 02:49:00,957 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 02:49:00,957 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 02:49:00,958 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 02:49:00,958 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:00,958 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 02:49:00,958 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:00,958 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:00,958 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,958 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:00,958 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:00,958 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,958 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 02:49:00,958 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:00,958 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:00,959 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,959 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:00,959 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:00,959 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:00,959 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:00,959 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:00,959 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:49:00,959 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:00,959 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:00,960 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 02:49:00,960 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:00,960 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:00,960 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:00,960 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:49:00,960 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:00,960 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:00,960 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:00,960 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 02:49:00,960 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:00,961 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 02:49:00,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:00,961 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:00,961 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 02:49:00,961 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 02:49:00,961 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 02:49:00,961 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:00,961 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 02:49:00,961 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 02:49:00,961 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:00,961 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:00,961 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 02:49:00,962 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:00,962 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 02:49:00,962 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:49:00,962 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 02:49:00,962 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 02:49:00,962 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:00,962 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 02:49:00,962 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 02:49:00,962 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:00,962 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 02:49:00,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 02:49:00,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:49:00,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 02:49:00,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 02:49:00,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:00,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 02:49:00,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 02:49:00,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:00,962 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]] - lastApplied: 1, lastIndex: 1 02:49:00,963 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 02:49:00,963 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 02:49:00,963 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:00,963 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:00,964 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 02:49:00,964 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 02:49:00,964 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 02:49:00,964 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 02:49:00,964 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 02:49:01,1 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 02:49:01,1 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,1 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 02:49:01,1 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,1 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,2 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:01,2 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 02:49:01,2 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:49:01,2 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:49:01,2 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 02:49:01,2 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 02:49:01,2 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,2 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:49:01,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:01,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 02:49:01,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:49:01,3 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:49:01,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 02:49:01,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 02:49:01,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:49:01,4 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 02:49:01,5 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 02:49:01,5 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 02:49:01,5 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 02:49:01,5 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:01,5 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:01,5 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 02:49:01,5 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 02:49:01,5 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 02:49:01,6 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 02:49:01,6 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,7 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,7 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 02:49:01,7 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,7 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 02:49:01,7 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,7 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 02:49:01,7 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 02:49:01,7 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 02:49:01,7 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,7 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,7 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 02:49:01,8 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 02:49:01,8 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 02:49:01,9 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 02:49:01,9 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 02:49:01,9 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,9 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,9 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,9 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:01,10 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 02:49:01,10 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,10 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,10 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,10 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:01,10 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:01,10 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,10 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:49:01,10 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,10 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,10 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,10 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 02:49:01,10 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,10 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,11 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,11 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,11 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,11 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,11 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,11 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:01,11 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,11 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:49:01,11 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,11 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,11 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 02:49:01,11 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 02:49:01,12 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]] - lastApplied: 1, lastIndex: 1 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,12 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,13 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 02:49:01,13 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 02:49:01,13 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 02:49:01,13 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 02:49:01,13 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:49:01,13 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 02:49:01,13 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 02:49:01,13 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 02:49:01,13 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,13 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 02:49:01,13 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 02:49:01,13 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,13 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 02:49:01,13 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 02:49:01,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 02:49:01,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:49:01,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 02:49:01,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 02:49:01,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 02:49:01,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 02:49:01,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 02:49:01,40 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 02:49:01,41 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,41 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 02:49:01,41 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,41 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 02:49:01,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,41 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 02:49:01,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,42 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,42 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:01,42 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 02:49:01,42 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:49:01,42 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:49:01,42 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 02:49:01,42 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 02:49:01,43 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,43 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:49:01,43 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:01,43 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 02:49:01,43 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:49:01,43 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:49:01,43 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 02:49:01,43 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 02:49:01,43 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,43 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:49:01,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:01,43 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:01,43 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 02:49:01,43 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:49:01,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 02:49:01,43 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:49:01,43 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 02:49:01,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 02:49:01,43 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 02:49:01,43 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 02:49:01,43 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 02:49:01,43 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:49:01,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 02:49:01,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:49:01,44 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 02:49:01,44 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 02:49:01,44 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 02:49:01,44 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 02:49:01,44 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 02:49:01,44 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 02:49:01,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 02:49:01,45 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 02:49:01,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:01,45 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:01,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:01,45 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:01,45 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 02:49:01,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 02:49:01,45 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 02:49:01,45 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 02:49:01,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 02:49:01,45 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 02:49:01,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,45 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 02:49:01,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 02:49:01,45 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 02:49:01,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,45 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 02:49:01,46 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 02:49:01,46 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 02:49:01,46 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 02:49:01,46 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,46 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,46 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 02:49:01,46 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,47 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,47 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 02:49:01,47 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 02:49:01,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,47 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 02:49:01,47 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 02:49:01,47 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 02:49:01,47 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,48 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 02:49:01,48 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,48 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,48 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,47 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 02:49:01,48 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:01,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,48 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,48 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 02:49:01,48 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,48 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 02:49:01,48 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,48 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 02:49:01,48 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,48 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,48 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,48 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 02:49:01,48 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:01,48 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 02:49:01,49 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 02:49:01,49 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,49 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,49 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,49 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:01,49 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,49 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 02:49:01,49 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 02:49:01,49 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 02:49:01,49 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 02:49:01,49 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 02:49:01,50 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:01,50 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,50 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:49:01,50 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,50 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,50 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 02:49:01,50 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,50 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,50 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 02:49:01,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:01,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:49:01,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,51 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 02:49:01,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 02:49:01,51 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:01,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:49:01,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 02:49:01,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,51 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,51 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,51 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:01,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 02:49:01,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 02:49:01,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 02:49:01,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 02:49:01,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 02:49:01,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 02:49:01,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,51 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,52 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 02:49:01,52 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,52 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]] - lastApplied: 1, lastIndex: 1 02:49:01,52 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 02:49:01,52 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,52 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,52 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 02:49:01,52 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 02:49:01,52 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 02:49:01,53 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 02:49:01,53 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 02:49:01,53 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 02:49:01,53 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 02:49:01,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:49:01,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 02:49:01,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 02:49:01,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 02:49:01,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 02:49:01,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,53 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 02:49:01,53 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:49:01,53 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 02:49:01,53 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 02:49:01,53 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 02:49:01,53 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,53 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 02:49:01,53 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 02:49:01,53 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,53 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 02:49:01,54 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,54 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 02:49:01,54 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 02:49:01,54 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 02:49:01,54 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 02:49:01,54 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:01,54 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,54 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 02:49:01,54 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,54 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,54 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 02:49:01,54 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,54 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 02:49:01,54 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,54 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 02:49:01,55 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] 02:49:01,55 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 02:49:01,55 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,55 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,55 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]] - lastApplied: 1, lastIndex: 1 02:49:01,55 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 02:49:01,55 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,55 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,55 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,55 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 02:49:01,55 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:49:01,55 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 02:49:01,55 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 02:49:01,55 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,55 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 02:49:01,55 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 02:49:01,55 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,56 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 02:49:01,55 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:01,56 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 02:49:01,56 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 02:49:01,56 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 02:49:01,56 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 02:49:01,56 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 02:49:01,56 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 02:49:01,56 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 02:49:01,56 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 02:49:01,56 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#296550892] 02:49:01,56 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:49:01,56 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#596402182] 02:49:01,56 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#596402182] 02:49:01,57 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 02:49:01,57 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e]]] 02:49:01,57 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,57 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e] is present: existingEntryTerm: 4 02:49:01,57 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 02:49:01,57 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 02:49:01,57 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 02:49:01,57 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,57 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 02:49:01,57 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 02:49:01,57 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 02:49:01,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 02:49:01,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 02:49:01,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 02:49:01,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 02:49:01,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@48b1e79e 02:49:01,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 02:49:01,58 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 02:49:01,58 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,58 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,58 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 02:49:01,58 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 02:49:01,59 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 02:49:01,59 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 02:49:01,59 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 02:49:01,59 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 02:49:01,59 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:49:01,59 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 02:49:01,59 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,60 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 02:49:01,60 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,60 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:49:01,60 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 02:49:01,60 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 02:49:01,60 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,60 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,60 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,60 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 02:49:01,60 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 02:49:01,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:49:01,60 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=663a1cbf-697d-4e02-9acf-8d3600cb2b46}, logIndex: 2, isSendImmediate: true 02:49:01,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 02:49:01,60 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]], state: Idle 02:49:01,60 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:49:01,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 02:49:01,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,60 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,60 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:49:01,60 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,60 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 02:49:01,60 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=bdfa8d4c-526b-432a-b07d-e32da4607abe}, logIndex: 2, isSendImmediate: true 02:49:01,60 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 02:49:01,60 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 02:49:01,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,60 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 02:49:01,60 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,60 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,60 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,60 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,60 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,60 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,60 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,60 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=0b7891cf-19e0-4802-9491-0d1de9a65e26}, logIndex: 2, isSendImmediate: true 02:49:01,60 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 02:49:01,60 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=e2048e25-0533-4d25-a956-4f56d47bf6ee}, logIndex: 2, isSendImmediate: true 02:49:01,61 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 02:49:01,61 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,61 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,61 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,61 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,61 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,61 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,61 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,61 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,61 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,61 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,61 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 02:49:01,61 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,61 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 02:49:01,61 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 02:49:01,61 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 02:49:01,61 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,61 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 02:49:01,61 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 02:49:01,61 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 02:49:01,61 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,61 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,61 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 02:49:01,61 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 02:49:01,61 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 02:49:01,61 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,62 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,62 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,62 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:49:01,62 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,62 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 02:49:01,63 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,64 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 02:49:01,64 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 02:49:01,64 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,64 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,64 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,64 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,64 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:49:01,64 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,64 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,64 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,64 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:49:01,64 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,64 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,65 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 02:49:01,65 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 02:49:01,65 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,65 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,65 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,65 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,65 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,65 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 02:49:01,65 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,65 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,65 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,65 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,66 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 02:49:01,66 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,66 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,66 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 02:49:01,66 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,66 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,66 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 02:49:01,66 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,66 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,66 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 02:49:01,66 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 02:49:01,66 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 02:49:01,66 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 02:49:01,66 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,67 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 02:49:01,67 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 02:49:01,67 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:01,67 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,66 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 02:49:01,67 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 02:49:01,67 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,67 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 02:49:01,67 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,67 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 02:49:01,67 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 02:49:01,67 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 02:49:01,67 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:01,68 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#296550892] 02:49:01,68 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:49:01,68 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#596402182] 02:49:01,68 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 02:49:01,68 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#596402182] 02:49:01,68 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#296550892] 02:49:01,68 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 02:49:01,68 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-298411493] 02:49:01,68 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-298411493] 02:49:01,69 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-378061802] 02:49:01,69 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 02:49:01,69 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1623182374] 02:49:01,69 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1623182374] 02:49:01,69 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-378061802] 02:49:01,69 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 02:49:01,69 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#710654609] 02:49:01,69 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#710654609] 02:49:01,69 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1593962586] 02:49:01,69 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:49:01,69 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-828214800] 02:49:01,69 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-828214800] 02:49:01,69 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1593962586] 02:49:01,70 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 02:49:01,70 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1611441101] 02:49:01,70 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1611441101] 02:49:01,70 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-943847062] 02:49:01,70 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 02:49:01,70 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-134984863] 02:49:01,70 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-134984863] 02:49:01,70 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-943847062] 02:49:01,70 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 02:49:01,70 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#924397661] 02:49:01,70 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#924397661] 02:49:01,70 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1010490443] 02:49:01,70 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:49:01,70 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-659837958] 02:49:01,70 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-659837958] 02:49:01,70 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1010490443] 02:49:01,70 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 02:49:01,70 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#736259823] 02:49:01,70 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#736259823] 02:49:01,70 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-279767169] 02:49:01,70 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 02:49:01,70 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#795148223] 02:49:01,70 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#795148223] 02:49:01,71 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-279767169] 02:49:01,71 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 02:49:01,71 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1503592257] 02:49:01,71 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1503592257] 02:49:01,71 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#296550892] 02:49:01,71 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:49:01,71 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#596402182] 02:49:01,71 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#596402182] 02:49:01,72 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 02:49:01,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,72 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 02:49:01,72 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 02:49:01,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 02:49:01,73 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,73 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 02:49:01,73 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 02:49:01,73 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 02:49:01,73 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 02:49:01,73 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 02:49:01,73 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,73 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,73 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 02:49:01,73 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 02:49:01,73 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 02:49:01,74 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,74 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,74 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 02:49:01,74 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 02:49:01,74 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 02:49:01,74 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 02:49:01,75 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:49:01,75 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:49:01,77 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 02:49:01,79 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:01,79 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:01,79 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:01,79 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:01,81 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:49:01,88 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 02:49:01,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,89 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 02:49:01,89 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 02:49:01,89 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 02:49:01,89 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,89 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 02:49:01,89 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,89 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 02:49:01,89 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 02:49:01,89 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 02:49:01,89 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 02:49:01,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,90 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 02:49:01,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 02:49:01,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 02:49:01,90 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 02:49:01,90 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:01,90 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 02:49:01,90 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 02:49:01,90 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 02:49:01,91 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:49:01,91 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:49:01,671 PM [cluster-test-shard-dispatcher-22] [WARN] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 02:49:01,671 PM [cluster-test-shard-dispatcher-34] [WARN] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 02:49:01,681 PM [cluster-test-shard-dispatcher-22] [WARN] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 02:49:01,681 PM [cluster-test-shard-dispatcher-34] [WARN] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 02:49:01,881 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 02:49:02,91 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [7909934857701572934] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:49:02,111 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-8196724518297213337] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:49:02,114 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:02,114 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:02,117 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:49:02.119] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:49:02,124 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 02:49:02,125 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:02,125 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:02,125 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 02:49:02,125 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 02:49:02,125 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 02:49:02,125 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 02:49:02,125 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 02:49:02,125 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@72982c2a 02:49:02,125 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2c522e56 02:49:02,125 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@72982c2a 02:49:02,125 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2c522e56 02:49:02,125 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 02:49:02,125 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 02:49:02,125 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 02:49:02,125 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 02:49:02,125 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:02,125 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 02:49:02,125 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:02,125 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:02,125 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 02:49:02,125 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:02,561 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 02:49:02,561 PM [cluster-test-shard-dispatcher-34] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 436.1 ms 02:49:02,561 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 02:49:02,561 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 02:49:02,561 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 02:49:02,562 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:02,562 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 02:49:02,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 02:49:02,651 PM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 526.4 ms 02:49:02,651 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 02:49:02,652 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 02:49:02,652 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 02:49:02,652 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 02:49:02,652 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 02:49:02,653 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:49:02,653 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:49:02,653 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 02:49:02,654 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:02,654 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:02,654 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 02:49:02,654 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 02:49:02,654 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 02:49:02,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 02:49:02,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 02:49:02,654 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@96c3632 02:49:02,654 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@96c3632 02:49:02,654 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 02:49:02,654 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 02:49:02,654 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 02:49:02,654 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:02,654 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 02:49:02,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@69ea75f1 02:49:02,654 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:02,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@69ea75f1 02:49:02,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 02:49:02,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 02:49:02,655 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:02,655 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 02:49:02,971 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 02:49:02,971 PM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 317.3 ms 02:49:02,972 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 02:49:02,972 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 02:49:02,972 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 02:49:02,972 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:02,973 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 02:49:03,251 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 02:49:03,251 PM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 596.8 ms 02:49:03,251 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 02:49:03,251 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 02:49:03,251 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 02:49:03,252 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 02:49:03,252 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 02:49:03,252 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:49:03,252 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:49:03,254 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:49:03,254 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:49:03,257 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:49:03.258] [cluster-test-akka.actor.internal-dispatcher-5] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:49:03,272 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:49:03,280 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [962540156404128057] 02:49:03,280 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:49:03,282 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:49:03,282 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:49:03,285 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:49:03,285 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:49:03,298 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:03,299 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:03,301 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 02:49:03,302 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:49:03,302 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:49:03,302 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:49:03,304 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-394722554] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,308 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:49:03,308 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:49:03,308 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,308 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:03,308 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:03,308 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,308 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:03,312 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 02:49:03,312 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@200f718f 02:49:03,312 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 02:49:03,313 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:03,313 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,313 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:03,313 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,313 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,313 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica on shard member-1-shard-cars-config_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-6438292] 02:49:03,314 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:03,314 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:03,314 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:03,314 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica on shard member-1-shard-people-config_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#-204526673] 02:49:03,314 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 02:49:03,314 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-6438292] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,314 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveShardLeaderReplica, persistent : true 02:49:03,315 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 02:49:03,315 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#1177416548 created and ready for shard:member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,315 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-1930855947] 02:49:03,315 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 02:49:03,315 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-1884621800] 02:49:03,315 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:03,318 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#-204526673] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,319 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 02:49:03,320 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#890515878 created and ready for shard:member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:03,321 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:49:03,326 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:03,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:03,326 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:03,326 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:03,326 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,326 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 02:49:03,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 02:49:03,326 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 02:49:03,326 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#1177416548] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,326 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 02:49:03,326 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-1930855947] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,326 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-1884621800] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 02:49:03,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardLeaderReplica is initialized 02:49:03,326 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:49:03,326 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#890515878] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,326 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#1557426175] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,326 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,326 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 02:49:03,327 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,327 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:03,327 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,327 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,328 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 02:49:03,328 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@11b84053 02:49:03,329 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 02:49:03,329 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 02:49:03,329 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardLeaderReplica is initialized 02:49:03,329 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 02:49:03,329 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 02:49:03,329 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from null to Follower 02:49:03,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:03,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,330 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-1-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#200020094] 02:49:03,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,330 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 02:49:03,330 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 02:49:03,331 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 02:49:03,331 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#1047891985] 02:49:03,332 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-817674184 created and ready for shard:member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,332 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#824387878 created and ready for shard:member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,332 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#154178591] 02:49:03,332 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 02:49:03,335 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#200020094] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,335 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#1160395359] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,336 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-1-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#1160395359] 02:49:03,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:03,336 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:03,336 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 02:49:03,336 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is initialized 02:49:03,336 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:03,336 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:03,336 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,336 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 02:49:03,336 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 02:49:03,336 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardLeaderReplica is initialized 02:49:03,337 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 02:49:03,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 02:49:03,337 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 02:49:03,337 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 02:49:03,337 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 02:49:03,341 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 02:49:03,341 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 02:49:03,341 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:49:03,349 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [2541744170031998222] 02:49:03,361 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:49:03,363 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:49:03,363 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:49:03,364 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:49:03,364 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:49:03,373 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:03,375 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:03,377 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 02:49:03,390 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1350813933] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,404 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 02:49:03,404 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 02:49:03,404 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,405 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardLeaderReplica 02:49:03,405 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 02:49:03,407 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@24464eba 02:49:03,407 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 02:49:03,408 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:03,408 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,408 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:03,409 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 02:49:03,409 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:03,409 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardLeaderReplica 02:49:03,411 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 02:49:03,411 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-739309750] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,411 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#994766028] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,411 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#994766028 created and ready for shard:member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,412 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardLeaderReplica, persistent : true 02:49:03,412 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 02:49:03,412 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica#-1341767192] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,413 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#315291556 created and ready for shard:member-2-shard-people-config_testRemoveShardLeaderReplica 02:49:03,413 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#1552702749] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,413 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#315291556] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,415 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#377219836] 02:49:03,415 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#233217372] 02:49:03,415 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 02:49:03,416 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#377219836] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,416 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#233217372] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,416 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 02:49:03,418 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 02:49:03,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:03,418 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:03,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 02:49:03,418 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 02:49:03,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 02:49:03,418 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardLeaderReplica is initialized 02:49:03,419 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 02:49:03,419 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardLeaderReplica from null to Follower 02:49:03,419 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:03,419 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:03,419 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 02:49:03,419 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardLeaderReplica is initialized 02:49:03,419 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,419 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 02:49:03,419 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 02:49:03,419 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 02:49:03,419 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 02:49:03,420 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 02:49:03,420 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 02:49:03,425 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7d190e2a 02:49:03,425 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:03,425 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,425 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,426 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 02:49:03,426 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 02:49:03,427 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 02:49:03,427 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 02:49:03,427 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#768535504 created and ready for shard:member-2-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,427 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-1624464297] 02:49:03,427 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica#-48700857] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,427 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica#-362754319] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,427 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#2080231545 created and ready for shard:member-2-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,427 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-660755765] 02:49:03,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:03,429 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:03,429 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:03,429 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:03,429 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,429 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 02:49:03,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,429 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 02:49:03,429 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 02:49:03,429 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 02:49:03,429 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardLeaderReplica is initialized 02:49:03,429 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 02:49:03,429 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 02:49:03,429 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica is initialized 02:49:03,429 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 02:49:03,429 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 02:49:03,429 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 02:49:03,429 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 02:49:03,434 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:49:03,441 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:49:03,443 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:49:03,443 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:03,443 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,443 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,443 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:03,443 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 02:49:03,443 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,443 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:03,443 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:03,443 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,443 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,443 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,443 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,443 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,443 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,452 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-1436203906037360145] 02:49:03,452 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 02:49:03,455 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:49:03,455 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 02:49:03,456 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:49:03,456 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:49:03,463 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:03,464 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:03,470 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 02:49:03,475 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#462651659] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,476 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 02:49:03,486 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 02:49:03,487 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 02:49:03,487 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,488 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardLeaderReplica 02:49:03,488 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 02:49:03,489 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:49:03,490 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:03,490 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,490 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:03,491 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@501df4dd 02:49:03,491 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 02:49:03,491 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:03,492 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:03,492 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,492 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:03,493 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 02:49:03,493 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 02:49:03,493 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica on shard member-3-shard-cars-config_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#849086913] 02:49:03,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardLeaderReplica 02:49:03,494 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveShardLeaderReplica, persistent : true 02:49:03,494 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 02:49:03,494 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:03,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 02:49:03,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica on shard member-3-shard-people-config_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica#785630603] 02:49:03,494 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 02:49:03,495 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#849086913] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,495 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 02:49:03,496 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#-494711013 created and ready for shard:member-3-shard-people-config_testRemoveShardLeaderReplica 02:49:03,496 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#1044483788] 02:49:03,496 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#940820711 created and ready for shard:member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,496 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica#785630603] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,497 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#872319355] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,497 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#-494711013] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,497 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#1044483788] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,497 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#940820711] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#530473084] 02:49:03,497 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:03,497 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:03,497 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:03,497 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:03,497 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#530473084] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,497 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,497 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 02:49:03,497 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:03,498 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 02:49:03,498 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardLeaderReplica is initialized 02:49:03,498 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 02:49:03,498 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardLeaderReplica is initialized 02:49:03,498 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 02:49:03,501 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,501 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 02:49:03,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 02:49:03,501 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 02:49:03,501 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 02:49:03,501 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 02:49:03,501 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardLeaderReplica from null to Follower 02:49:03,501 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 02:49:03,502 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,502 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,502 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:03,502 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,502 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,506 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 02:49:03,506 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@423d11ce 02:49:03,506 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 02:49:03,507 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:03,507 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,507 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 02:49:03,507 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 02:49:03,508 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#639513259] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,508 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#191533316 created and ready for shard:member-3-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,508 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 02:49:03,508 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-1109720094] 02:49:03,508 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#191533316] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:03,508 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-3-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#639513259] 02:49:03,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:03,508 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:03,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 02:49:03,508 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,509 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 02:49:03,509 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 02:49:03,509 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 02:49:03,509 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#982931273 created and ready for shard:member-3-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,509 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#1511284673] 02:49:03,510 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,510 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,510 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-3-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica#1439874815] 02:49:03,510 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:03,510 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:03,510 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,510 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 02:49:03,510 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,510 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 02:49:03,510 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 02:49:03,510 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica is initialized 02:49:03,510 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 02:49:03,510 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 02:49:03,510 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardLeaderReplica is initialized 02:49:03,510 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 02:49:03,510 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 02:49:03,510 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 02:49:03,657 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:49:03,657 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 02:49:03,657 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 02:49:03,657 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:03,657 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 02:49:03,657 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:49:03,657 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,657 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 02:49:03,658 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 02:49:03,658 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Follower to Candidate 02:49:03,677 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:49:03,677 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 02:49:03,677 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:49:03,677 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 02:49:03,677 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:49:03,677 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 02:49:03,677 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 02:49:03,677 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:03,677 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 02:49:03,677 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica, member-3-shard-cars-config_testRemoveShardLeaderReplica] 02:49:03,677 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 02:49:03,677 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:03,677 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:49:03,677 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:03,677 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,677 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 02:49:03,677 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 02:49:03,677 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 02:49:03,678 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:49:03,678 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:49:03,678 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,678 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:03,678 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 02:49:03,678 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 02:49:03,678 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 02:49:03,678 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 02:49:03,678 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 02:49:03,678 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Follower to Candidate 02:49:03,678 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Follower to Candidate 02:49:03,678 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 02:49:04,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 02:49:04,18 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,18 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 02:49:04,67 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 02:49:04,68 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:04,68 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 02:49:04,88 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 02:49:04,88 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,88 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 02:49:04,97 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 02:49:04,97 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,97 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 02:49:04,299 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 02:49:04,299 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:49:04,299 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:04,299 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,299 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,299 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 02:49:04,299 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:04,299 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:04,300 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:04,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:04,300 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:04,300 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:04,307 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:04,307 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:04,307 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:04,307 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,307 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,308 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 02:49:04,308 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 02:49:04,379 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 02:49:04,379 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,379 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 02:49:04,379 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,380 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,381 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:04,381 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 02:49:04,381 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:04,381 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:04,382 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:04,382 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 02:49:04,382 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:04,382 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:04,382 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 02:49:04,382 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 02:49:04,382 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:04,382 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,382 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:04,383 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:04,383 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,383 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:04,384 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:49:04,384 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:49:04,384 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 02:49:04,384 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,384 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,384 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:49:04,384 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:49:04,384 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 02:49:04,384 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 02:49:04,384 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,384 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Candidate to Leader 02:49:04,385 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,386 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,386 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 02:49:04,387 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 02:49:04,387 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,387 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,388 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,388 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 02:49:04,388 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 02:49:04,388 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,392 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:04,392 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,392 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,392 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 02:49:04,392 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 02:49:04,392 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:04,392 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,392 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,393 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 02:49:04,393 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 02:49:04,393 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,393 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 02:49:04,393 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:04,393 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,393 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,393 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,393 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,393 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:04,393 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,393 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,393 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,393 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,393 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,393 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,407 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 02:49:04,407 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:04,408 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 02:49:04,408 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testRemoveShardLeaderReplica 02:49:04,408 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testRemoveShardLeaderReplica 02:49:04,409 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:04,409 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 02:49:04,409 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:04,409 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:04,410 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 02:49:04,410 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:04,410 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:04,410 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:04,410 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:04,410 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 02:49:04,410 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:04,410 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:04,410 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 02:49:04,410 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:04,411 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:04,411 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:04,411 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:49:04,411 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:49:04,411 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 02:49:04,411 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,411 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,411 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:49:04,412 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:49:04,412 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 02:49:04,412 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,412 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 02:49:04,412 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Candidate to Leader 02:49:04,413 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:04,413 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:04,413 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,414 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardLeaderReplica status sync done false 02:49:04,414 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardLeaderReplica status sync done false 02:49:04,414 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,414 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 02:49:04,414 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,414 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 02:49:04,415 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,474 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 02:49:04,497 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 02:49:04,497 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,497 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 02:49:04,497 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,498 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,499 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:04,499 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 02:49:04,499 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:04,499 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:04,499 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 02:49:04,500 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:04,500 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,500 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:04,500 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:04,500 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 02:49:04,500 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:04,500 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:04,500 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 02:49:04,500 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:04,500 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,501 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:49:04,501 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:49:04,501 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica, member-3-shard-cars-config_testRemoveShardLeaderReplica] 02:49:04,501 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,501 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,502 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:49:04,502 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:49:04,502 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 02:49:04,502 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,502 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 02:49:04,502 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 02:49:04,502 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 02:49:04,502 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} is now ready 02:49:04,502 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-394722554] 02:49:04,502 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:49:04,502 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-6438292] 02:49:04,502 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-6438292] 02:49:04,503 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1350813933] 02:49:04,500 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:04,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:49:04,503 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-739309750] 02:49:04,503 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-739309750] 02:49:04,505 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 02:49:04,505 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,505 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,505 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 02:49:04,505 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,505 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 02:49:04,505 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,505 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 02:49:04,507 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,507 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,507 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 02:49:04,507 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 02:49:04,507 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,507 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 02:49:04,507 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,507 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,508 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#462651659] 02:49:04,508 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 02:49:04,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:49:04,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:04,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 02:49:04,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:04,508 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:04,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 02:49:04,508 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#849086913] 02:49:04,508 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#849086913] 02:49:04,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:04,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,508 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:04,508 PM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-1, datastoreType Config 02:49:04,508 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-1}] 02:49:04,509 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:04,509 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 02:49:04,509 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:04,509 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:04,509 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 02:49:04,509 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:04,509 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,509 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:04,509 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 02:49:04,509 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-6438292, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4137656b] 02:49:04,510 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-6438292 for shard member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,510 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onRemoveServer: RemoveServer{serverId='member-1-shard-cars-config_testRemoveShardLeaderReplica'}, state: Idle 02:49:04,510 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]] 02:49:04,510 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 02:49:04,510 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 02:49:04,510 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=8ccd4d3d-fb3c-4035-b771-a1dc2e314f51}, logIndex: 0, isSendImmediate: true 02:49:04,510 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,510 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,510 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveShardLeaderReplica: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 02:49:04,510 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,510 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,510 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testRemoveShardLeaderReplica: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 02:49:04,510 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: Returning OK for operation RemoveServer{serverId='member-1-shard-cars-config_testRemoveShardLeaderReplica'} 02:49:04,510 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardLeaderReplica} for shard cars 02:49:04,510 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 02:49:04,511 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 02:49:04,511 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 02:49:04,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1350813933] 02:49:04,511 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:49:04,511 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:49:04,511 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-739309750] 02:49:04,511 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:49:04,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-739309750] 02:49:04,511 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 02:49:04,511 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,511 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,511 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:49:04,511 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:49:04,512 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,512 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 02:49:04,512 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 02:49:04,512 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Candidate to Leader 02:49:04,512 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 02:49:04,512 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,512 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done false 02:49:04,512 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 02:49:04,512 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Number of entries to be appended = 1 02:49:04,512 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:49:04,512 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 02:49:04,512 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 02:49:04,513 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 02:49:04,513 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 02:49:04,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Number of entries to be appended = 1 02:49:04,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:49:04,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 02:49:04,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 02:49:04,513 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,513 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardLeaderReplica status sync done false 02:49:04,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testRemoveShardLeaderReplica, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica, votingState=VOTING]] 02:49:04,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Log size is now 1 02:49:04,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:04,513 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 02:49:04,513 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:04,513 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,513 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 02:49:04,513 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 02:49:04,513 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 02:49:04,514 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testRemoveShardLeaderReplica, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica, votingState=VOTING]] 02:49:04,514 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Log size is now 1 02:49:04,514 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 02:49:04,514 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:04,514 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 02:49:04,514 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Setting last applied to 0 02:49:04,514 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 02:49:04,514 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 02:49:04,514 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,514 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 02:49:04,514 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 02:49:04,514 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 02:49:04,514 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,514 PM [cluster-test-shard-dispatcher-33] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] has been successfully replicated to a majority of followers 02:49:04,514 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 02:49:04,514 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 02:49:04,514 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-6438292] with 10000 ms timeout 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1 : Local Shard replica for shard cars has been removed 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: persisting the shard list [people] 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Successfully deleted prior snapshots 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Initiating leader transfer 02:49:04,515 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Commit index set to 0 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4b4d742a 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4b4d742a 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Attempting to transfer leadership 02:49:04,515 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:49:04,515 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Setting last applied to 0 02:49:04,515 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,515 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,515 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 02:49:04,515 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,516 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Commit index set to 0 02:49:04,516 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-config_testRemoveShardLeaderReplica, matchIndex: 0, lastIndex: 0, isVoting: true 02:49:04,516 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:49:04,516 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Follower's log matches - sending ElectionTimeout 02:49:04,516 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Setting last applied to 0 02:49:04,516 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 02:49:04,516 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,516 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 02:49:04,516 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,516 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leader transfer complete 02:49:04,516 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 02:49:04,517 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica] 02:49:04,517 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 02:49:04,517 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 02:49:04,517 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica] 02:49:04,518 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 02:49:04,518 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Received TimeoutNow - switching to Candidate 02:49:04,518 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica] 02:49:04,518 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=4, votedFor=member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,518 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 4 02:49:04,518 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=4, candidateId=member-3-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] to peer member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,518 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 4 02:49:04,518 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:04,518 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 02:49:04,518 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 02:49:04,518 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 02:49:04,519 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 02:49:04,519 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:04,519 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 02:49:04,519 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-3-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 02:49:04,519 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 4 in "RequestVote [term=4, candidateId=member-3-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 3 - updating term 02:49:04,519 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=4, votedFor=null 02:49:04,519 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=4, candidateId=member-3-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 3 02:49:04,519 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=4, votedFor=member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,519 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 02:49:04,520 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 02:49:04,520 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 02:49:04,520 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica] 02:49:04,520 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,520 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 02:49:04,520 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:49:04,520 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 02:49:04,520 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,520 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 02:49:04,520 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 02:49:04,520 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 02:49:04,520 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-3-shard-cars-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,520 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,521 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 02:49:04,521 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 02:49:04,521 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 02:49:04,521 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,521 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 02:49:04,564 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-739309750] 02:49:04,565 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1350813933] 02:49:04,565 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:49:04,565 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-739309750] 02:49:04,565 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-739309750] 02:49:04,565 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#462651659] 02:49:04,565 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:49:04,565 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#849086913] 02:49:04,565 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#849086913] 02:49:04,566 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-394722554] 02:49:04,566 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:49:04,566 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: Local shard cars not found - shards present: [people] 02:49:04,566 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 02:49:04,566 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,566 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardLeaderReplica 02:49:04,566 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 02:49:04,566 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 02:49:04,566 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onShutDown 02:49:04,566 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Initiating leader transfer 02:49:04,566 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 02:49:04,567 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveShardLeaderReplica 02:49:04,567 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@423681d5 02:49:04,567 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-3-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@423681d5 02:49:04,567 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 02:49:04,567 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Attempting to transfer leadership 02:49:04,567 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 02:49:04,567 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,568 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testRemoveShardLeaderReplica] 02:49:04,568 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 02:49:04,569 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testRemoveShardLeaderReplica, matchIndex: 0, lastIndex: 0, isVoting: true 02:49:04,569 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Follower's log matches - sending ElectionTimeout 02:49:04,569 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,569 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leader transfer complete 02:49:04,569 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 02:49:04,570 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Received TimeoutNow - switching to Candidate 02:49:04,570 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testRemoveShardLeaderReplica] 02:49:04,570 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=5, votedFor=member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,570 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 5 02:49:04,570 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] to peer member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,570 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 5 02:49:04,570 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:04,571 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 02:49:04,571 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 02:49:04,571 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 02:49:04,571 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 02:49:04,572 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:04,572 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Cluster state: [] 02:49:04,572 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 02:49:04,572 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3]" message is greater than leader's term 4 - switching to Follower 02:49:04,572 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=5, votedFor=null 02:49:04,572 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leadership transfer in progress - processing RequestVote 02:49:04,572 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 0, lastTerm: 3 02:49:04,572 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=5, votedFor=member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,572 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 02:49:04,572 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 5 02:49:04,573 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:04,573 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 02:49:04,573 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 02:49:04,573 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 02:49:04,573 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 02:49:04,573 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 02:49:04,574 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 02:49:04,574 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testRemoveShardLeaderReplica] 02:49:04,574 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,574 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 02:49:04,574 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:49:04,574 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 02:49:04,574 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 02:49:04,574 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,574 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 02:49:04,574 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 02:49:04,575 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-2-shard-cars-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,575 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: leader changed to member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,575 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 02:49:04,575 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to member-2-shard-cars-config_testRemoveShardLeaderReplica in 8.319 ms 02:49:04,575 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: leader transfer succeeded - sending PoisonPill 02:49:04,575 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 02:49:04,575 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 02:49:04,575 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,575 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 02:49:04,575 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 02:49:04,575 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 02:49:04,575 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 02:49:04,576 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,576 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 02:49:04,576 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,576 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,576 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 02:49:04,576 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 02:49:04,576 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 02:49:04,576 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 02:49:04,577 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 02:49:04,577 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 02:49:04,577 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 02:49:04,577 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 02:49:04,577 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 02:49:04,578 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:49:04,578 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:49:04,582 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:04,583 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:04,583 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:04,583 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:04,584 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:49:04,595 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 02:49:04,595 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:04,595 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardLeaderReplica 02:49:04,595 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 02:49:04,595 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 02:49:04,595 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Initiating leader transfer 02:49:04,595 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onShutDown 02:49:04,595 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 02:49:04,595 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1237499b 02:49:04,595 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1237499b 02:49:04,595 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 02:49:04,595 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Attempting to transfer leadership 02:49:04,595 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:04,595 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardLeaderReplica 02:49:04,595 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 02:49:04,699 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done true 02:49:04,700 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 313, lastApplied : -1, commitIndex : -1 02:49:04,829 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done true 02:49:04,830 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 02:49:05,21 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:05,140 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Cannot append entries because sender's term 3 is less than 5 02:49:05,141 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 5 in "AppendEntriesReply [term=5, success=false, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5]" message is greater than leader's term 3 - switching to Follower 02:49:05,141 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=5, votedFor=null 02:49:05,141 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 5 02:49:05,141 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 02:49:05,141 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 02:49:05,141 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 02:49:05,141 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:05,141 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 02:49:05,141 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 02:49:05,148 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:49:05,318 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 02:49:05,340 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:49:05,471 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 02:49:05,594 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-1436203906037360145] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:49:05,597 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-1436203906037360145] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:49:05,662 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:49:05,790 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:05,980 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:49:06,111 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:06,300 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:06,429 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:49:06,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader not elected in time 02:49:06,527 PM [cluster-test-shard-dispatcher-33] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to null in 2.012 s 02:49:06,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer succeeded - sending PoisonPill 02:49:06,528 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 02:49:06,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 02:49:06,528 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-6438292] 02:49:06,620 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:06,750 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:06,941 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:49:07,71 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:07,260 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 02:49:07,389 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:49:07,579 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:49:07,711 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:49:07,900 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:08,30 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:49:08,221 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:49:08,350 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:08,540 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:49:08,670 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:08,861 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:08,990 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:09,180 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:09,300 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 309, lastApplied : -1, commitIndex : -1 02:49:09,487 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 02:49:09,488 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 02:49:09,488 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:09,488 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:09,488 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 02:49:09,488 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:49:09,488 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 02:49:09,489 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 02:49:09,490 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 310, lastApplied : -1, commitIndex : -1 02:49:09,490 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - IsolatedLeader member-1-shard-cars-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 02:49:09,490 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 02:49:09,490 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:09,490 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:09,490 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 02:49:09,491 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:49:09,491 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 02:49:09,491 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 02:49:09,491 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 02:49:09,508 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 02:49:09,508 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 02:49:09,508 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:09,508 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:09,508 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 02:49:09,508 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:49:09,508 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 02:49:09,508 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 02:49:09,607 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 02:49:09,607 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 02:49:09,607 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:09,607 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:09,607 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 02:49:09,608 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:49:09,608 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 02:49:09,608 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 02:49:09,610 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 310, lastApplied : -1, commitIndex : -1 02:49:09,610 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - IsolatedLeader member-1-shard-people-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 02:49:09,610 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 02:49:09,610 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:09,611 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:09,611 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 02:49:09,611 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:49:09,611 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 02:49:09,611 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 02:49:09,611 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 02:49:09,810 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 02:49:09,930 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 02:49:10,130 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:10,249 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:10,418 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 02:49:10,419 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:10,419 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:10,420 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:49:10,423 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:10,423 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:10,449 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:10,494 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 02:49:10,569 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:10,770 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:10,890 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:11,91 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:49:11,210 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:11,410 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:49:11,530 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:11,730 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:11,850 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:11,929 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:49:12,50 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:12,170 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:12,371 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 02:49:12,373 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 02:49:12,489 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:12,690 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:49:12,810 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:13,10 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:13,129 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:13,330 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:13,450 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 02:49:13,649 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 02:49:13,770 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 02:49:13,843 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer aborted 02:49:13,843 PM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: Failed to transfer leadership in 9.248 s 02:49:13,843 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In unpauseLeader 02:49:13,844 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer failed - sending PoisonPill 02:49:13,844 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leadership transfer expired 02:49:13,844 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 02:49:13,844 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 02:49:13,845 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 02:49:13,845 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 02:49:13,845 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 02:49:13,845 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 02:49:13,845 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardLeaderReplica 02:49:13,846 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 02:49:13,846 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 02:49:13,846 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 02:49:13,846 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 02:49:13,846 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 02:49:13,846 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 02:49:13,846 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 02:49:13,846 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 02:49:13,846 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 02:49:13,848 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:49:13,848 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:49:14,608 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 02:49:14,608 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 02:49:14,608 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:14,608 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:14,608 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 02:49:14,608 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:49:14,608 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 02:49:14,608 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 02:49:14,728 PM [cluster-test-shard-dispatcher-34] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 02:49:14,728 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 02:49:14,728 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:14,728 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:14,728 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 02:49:14,728 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 02:49:14,728 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 02:49:14,728 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 02:49:14,866 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:14,867 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:14,868 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [2541744170031998222] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:49:14,868 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:49:14.870] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:49:14,875 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 02:49:14,875 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:14,876 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 02:49:14,876 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onShutDown 02:49:14,876 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveShardLeaderReplica 02:49:14,876 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 02:49:14,876 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:49:14,876 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:49:14,877 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 02:49:14,877 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:14,877 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:14,877 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 02:49:14,877 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 02:49:14,877 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 02:49:14,877 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 02:49:14,877 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 02:49:14,877 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 02:49:14,877 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 02:49:14,877 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:49:14,878 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:49:14,878 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:49:14,878 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:49:14,880 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:49:14.881] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:49:14,897 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 02:49:14,904 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7459395839669834370] 02:49:14,904 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:49:14,907 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:49:14,907 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:49:14,907 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:49:14,907 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:49:14,917 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:49:14,917 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:49:14,917 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:49:14,917 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:14,919 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:14,921 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,930 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-554170955] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:14,932 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:49:14,933 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:49:14,933 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,933 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,934 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:14,934 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,934 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,934 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 02:49:14,935 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@71820bc3 02:49:14,935 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 02:49:14,935 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:14,935 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,935 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:14,936 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,936 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,936 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1775306728] 02:49:14,936 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,936 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,936 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,936 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#-409683913] 02:49:14,936 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 02:49:14,936 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 02:49:14,937 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 02:49:14,937 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 02:49:14,937 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:14,939 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1775306728] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:14,939 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#-409683913] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:14,939 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,941 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:49:14,941 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#132359968 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,941 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#132359968] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:14,941 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-8711304 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,941 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-2145134916] 02:49:14,941 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-872556486] 02:49:14,942 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard#-1955346943] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:14,942 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-8711304] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:14,942 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-2145134916] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:14,942 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-872556486] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:14,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:14,953 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:14,953 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 02:49:14,953 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 02:49:14,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:14,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 02:49:14,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 02:49:14,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 02:49:14,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,953 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 02:49:14,953 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:49:14,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:14,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 02:49:14,954 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 02:49:14,954 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 02:49:14,954 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:49:14,954 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:14,954 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,954 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:14,954 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,954 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 02:49:14,954 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 02:49:14,954 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:49:14,955 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 02:49:14,955 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:14,955 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,955 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,955 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 02:49:14,955 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 02:49:14,955 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 02:49:14,955 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 02:49:14,955 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:14,955 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 02:49:14,955 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 02:49:14,955 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 02:49:14,955 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,955 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 02:49:14,955 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:49:14,955 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:14,955 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 02:49:14,956 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 02:49:14,956 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 02:49:14,956 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 02:49:14,956 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 02:49:14,956 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:49:14,956 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 02:49:14,956 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 02:49:14,957 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,957 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 02:49:14,957 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 02:49:14,957 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 02:49:14,957 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 02:49:14,957 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,957 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 02:49:14,957 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3faa7bd7 02:49:14,957 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 02:49:14,957 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 02:49:14,957 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForSingleNodeShard is ready 02:49:14,957 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 02:49:14,957 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:14,957 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,957 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} is now ready 02:49:14,958 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,958 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,958 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#871422017] 02:49:14,958 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,958 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,958 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,958 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#-1015868666] 02:49:14,959 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 02:49:14,959 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#-1015868666] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:14,959 PM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard cars: {member-1=false} 02:49:14,959 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-1=false}] 02:49:14,959 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 02:49:14,959 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:49:14,960 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-103408280] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:14,960 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 02:49:14,960 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,960 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]], state: Idle 02:49:14,960 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:49:14,960 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Returning INVALID_REQUEST for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] 02:49:14,960 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=INVALID_REQUEST, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard} from local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,961 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1223625578 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,961 PM [cluster-test-akka.actor.default-dispatcher-14] [ERROR] ClusterAdminRpcService - Failed to change member voting states for shard cars java.lang.IllegalArgumentException: The requested voting state change for shard cars is invalid. At least one member must be voting at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$11.onComplete(ShardManager.java:1558) at akka.dispatch.OnComplete.internal(Future.scala:300) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 02:49:14,961 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 02:49:14,961 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-554170955] 02:49:14,961 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-770271873] 02:49:14,961 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#26073494] 02:49:14,961 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-103408280 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,961 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:49:14,961 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1775306728] 02:49:14,961 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1775306728] 02:49:14,961 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:14,961 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:14,962 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 02:49:14,962 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:14,962 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 02:49:14,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 02:49:14,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 02:49:14,962 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 02:49:14,962 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 02:49:14,962 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 02:49:14,962 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,962 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,962 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 02:49:14,962 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:49:14,962 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:14,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:14,962 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,962 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:14,962 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 02:49:14,962 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@27dbce85 02:49:14,962 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@27dbce85 02:49:14,962 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 02:49:14,962 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 02:49:14,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:14,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 02:49:14,962 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 02:49:14,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 02:49:14,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 02:49:14,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,962 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 02:49:14,962 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:49:14,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:14,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 02:49:14,962 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 02:49:14,962 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,962 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Aborting 0 pending queued transactions 02:49:14,962 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 02:49:14,963 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 02:49:14,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 02:49:14,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 02:49:14,963 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:49:14,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 02:49:14,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 02:49:14,963 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 02:49:14,963 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 02:49:14,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 02:49:14,963 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 02:49:14,963 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 02:49:14,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 02:49:14,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 02:49:14,963 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 02:49:14,963 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 02:49:14,963 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,963 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 02:49:14,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 02:49:14,963 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 02:49:14,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@3889f8f9 02:49:14,963 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@3889f8f9 02:49:14,964 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,964 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Aborting 0 pending queued transactions 02:49:14,964 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:49:14,964 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:49:14,964 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 02:49:14,964 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForSingleNodeShard is ready 02:49:14,964 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 02:49:14,964 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,964 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,965 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 02:49:14,965 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 02:49:14,965 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@355210a7 02:49:14,965 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@100b7f6e 02:49:14,965 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@355210a7 02:49:14,965 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@100b7f6e 02:49:14,965 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 02:49:14,965 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,965 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Aborting 0 pending queued transactions 02:49:14,965 PM [cluster-test-shard-dispatcher-24] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 02:49:14,965 PM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Aborting 0 pending queued transactions 02:49:14,965 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:49:14,965 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:49:14,966 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:49:14,966 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:49:14,968 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:49:14,981 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:49:14,993 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [9030840538118137679] 02:49:14,993 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 02:49:14,995 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 02:49:14,995 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 02:49:14,996 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:49:14,996 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:49:15,9 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:15,11 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 02:49:15,11 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 02:49:15,11 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 02:49:15,12 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:15,14 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 02:49:15,18 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-310557986] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,19 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 02:49:15,20 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 02:49:15,20 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,20 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,20 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:15,20 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,20 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,22 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 02:49:15,22 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@210aa9ca 02:49:15,22 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 02:49:15,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:15,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,23 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:15,23 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-563325511] 02:49:15,23 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,24 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,24 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 02:49:15,24 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,24 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard on shard member-1-shard-people-config_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#1262683717] 02:49:15,24 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 02:49:15,25 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 02:49:15,25 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1718579476 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,25 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:15,25 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-1235005364] 02:49:15,25 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 02:49:15,25 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-563325511] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,26 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#848062452] 02:49:15,26 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#1262683717] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,26 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1718579476] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,26 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-1235005364] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,26 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#848062452] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,27 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1443972649 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,27 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1443972649] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,28 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 02:49:15,29 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#1669254976] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,29 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:15,29 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:15,29 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 02:49:15,29 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 02:49:15,29 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:15,29 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:15,29 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 02:49:15,29 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is initialized 02:49:15,29 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 02:49:15,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,30 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 02:49:15,30 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 02:49:15,30 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 02:49:15,30 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,30 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,30 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 02:49:15,30 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 02:49:15,30 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 02:49:15,30 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 02:49:15,30 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,31 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,31 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:15,32 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 02:49:15,31 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,32 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,33 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1d9fbf6d 02:49:15,33 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 02:49:15,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:15,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,34 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 02:49:15,34 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#2005889693] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,35 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 02:49:15,35 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#1618977753 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,35 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-1965154588] 02:49:15,35 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#1618977753] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,36 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#2005889693] 02:49:15,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,36 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 02:49:15,36 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-554482699] 02:49:15,37 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 02:49:15,37 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,37 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:15,37 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-1455453233] 02:49:15,37 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:15,37 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 02:49:15,37 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 02:49:15,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,37 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 02:49:15,37 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 02:49:15,37 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 02:49:15,37 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:15,37 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:15,37 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#752898009 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 02:49:15,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 02:49:15,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,38 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 02:49:15,38 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 02:49:15,38 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 02:49:15,40 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 02:49:15,49 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1902556325510241739] 02:49:15,50 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 02:49:15,51 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:49:15,52 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 02:49:15,52 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:49:15,52 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:49:15,64 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:15,67 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:15,69 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 02:49:15,75 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#879042657] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,76 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 02:49:15,82 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 02:49:15,82 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,82 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,84 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 02:49:15,91 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@222ac6f8 02:49:15,91 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 02:49:15,91 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:15,91 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,91 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:15,93 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:15,95 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 02:49:15,99 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 02:49:15,101 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:49:15,105 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,105 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:15,105 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,105 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,105 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,105 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,105 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-1164060352] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,106 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 02:49:15,106 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-1022513670] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,106 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-677924194 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,106 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-677924194] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,106 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 02:49:15,106 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 02:49:15,107 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 02:49:15,107 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 02:49:15,113 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard#-2071709830] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,113 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1430231653 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,113 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1430231653] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,113 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#-453079400] 02:49:15,113 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#-453079400] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,118 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:15,118 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:15,118 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:15,118 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:15,118 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,118 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,118 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,118 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 02:49:15,118 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 02:49:15,118 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 02:49:15,118 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard is initialized 02:49:15,118 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 02:49:15,119 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 02:49:15,119 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#622925320] 02:49:15,119 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#622925320] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,119 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,119 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,120 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 02:49:15,120 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,120 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 02:49:15,123 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 02:49:15,123 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 02:49:15,125 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 02:49:15,125 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,126 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,126 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:15,126 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,126 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,128 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 02:49:15,128 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1a083a08 02:49:15,128 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 02:49:15,129 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:15,129 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,129 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,129 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,129 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard#-1571662929] 02:49:15,129 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,129 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,129 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,129 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard#-2033882733] 02:49:15,129 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard#-1571662929] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,130 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard#-2033882733] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,130 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Shard created : member-2-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 02:49:15,130 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#-421075640 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,130 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#-1672432838] 02:49:15,131 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 02:49:15,131 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Shard created : member-2-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 02:49:15,131 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-673857595 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,131 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#536714631] 02:49:15,131 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 02:49:15,132 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:15,132 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:15,132 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 02:49:15,132 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 02:49:15,132 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:15,132 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:15,132 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 02:49:15,132 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 02:49:15,132 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,132 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,132 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,132 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 02:49:15,132 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 02:49:15,132 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 02:49:15,132 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,132 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,132 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 02:49:15,132 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,132 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 02:49:15,132 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 02:49:15,137 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 02:49:15,145 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4361887230769247356] 02:49:15,145 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 02:49:15,147 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:49:15,147 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 02:49:15,150 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 02:49:15,150 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 02:49:15,159 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:15,160 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:15,168 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 02:49:15,178 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 02:49:15,180 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#941759962] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,181 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 02:49:15,181 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,181 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,185 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 02:49:15,186 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@9405317 02:49:15,186 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 02:49:15,187 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:15,187 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,187 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,188 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:15,188 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 02:49:15,188 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 02:49:15,188 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 02:49:15,189 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 02:49:15,190 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 02:49:15,193 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-2060303948] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,193 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard#-872840070] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,193 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-1113121327] 02:49:15,193 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#217902011 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,193 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#1792627224 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,193 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-34726711] 02:49:15,193 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#217902011] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,193 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 02:49:15,195 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 02:49:15,200 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 02:49:15,203 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 02:49:15,204 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:15,204 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,204 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,204 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,204 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,206 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:15,206 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:15,208 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 02:49:15,209 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 02:49:15,210 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 02:49:15,212 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-1113121327] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,212 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#1792627224] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,212 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-34726711] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,212 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-750288220] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,212 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 02:49:15,212 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 02:49:15,212 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:15,212 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:15,212 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 02:49:15,212 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard is initialized 02:49:15,213 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,213 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,213 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,213 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 02:49:15,213 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 02:49:15,213 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 02:49:15,213 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 02:49:15,213 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,214 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,214 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 02:49:15,214 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,214 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,214 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@d43a617 02:49:15,214 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 02:49:15,214 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,214 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,214 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,214 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#1070098049] 02:49:15,214 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,215 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 02:49:15,215 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#1070098049] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,215 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Shard created : member-3-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 02:49:15,215 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-680106719 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,215 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#1346728316] 02:49:15,215 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#-76103311 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,215 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#1456056335] 02:49:15,215 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 02:49:15,215 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#1351562026] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 02:49:15,215 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 02:49:15,216 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:15,216 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 02:49:15,216 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:15,216 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 02:49:15,216 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,216 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,216 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,216 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,216 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 02:49:15,216 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 02:49:15,216 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,216 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,216 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,216 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#1351562026] 02:49:15,216 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,216 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 02:49:15,216 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 02:49:15,216 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 02:49:15,216 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 02:49:15,216 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 02:49:15,216 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 02:49:15,216 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 02:49:15,216 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 02:49:15,217 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,217 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,217 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,217 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 02:49:15,217 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 02:49:15,217 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 02:49:15,357 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:49:15,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 02:49:15,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testChangeMemberVotingStatusForShard, member-2-shard-people-config_testChangeMemberVotingStatusForShard] 02:49:15,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,358 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 02:49:15,359 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:49:15,359 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,359 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,359 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 02:49:15,359 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Follower to Candidate 02:49:15,377 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:49:15,377 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 02:49:15,377 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, member-2-shard-cars-oper_testChangeMemberVotingStatusForShard] 02:49:15,377 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,377 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 02:49:15,377 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:49:15,377 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,377 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,377 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 02:49:15,377 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 02:49:15,426 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:49:15,426 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 02:49:15,426 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testChangeMemberVotingStatusForShard, member-3-shard-cars-config_testChangeMemberVotingStatusForShard] 02:49:15,426 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,426 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 02:49:15,427 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:49:15,427 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,427 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,427 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 02:49:15,427 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Follower to Candidate 02:49:15,457 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 02:49:15,457 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 02:49:15,457 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testChangeMemberVotingStatusForShard, member-3-shard-people-oper_testChangeMemberVotingStatusForShard] 02:49:15,457 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,457 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 02:49:15,457 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 02:49:15,457 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 02:49:15,457 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 02:49:15,457 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 02:49:15,457 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 02:49:15,716 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 02:49:15,716 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:15,717 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 02:49:15,796 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 02:49:15,796 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:15,796 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 02:49:15,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 02:49:15,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:15,807 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 02:49:15,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 02:49:15,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:15,846 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 02:49:16,10 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 02:49:16,10 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 02:49:16,12 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:16,12 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:16,12 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:16,12 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:16,12 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,12 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:16,22 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:16,22 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:16,22 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:16,22 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,22 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,88 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:16,88 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,88 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,88 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,88 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,88 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:16,88 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,88 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,88 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,88 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,89 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,89 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,89 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 02:49:16,89 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,89 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,89 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,89 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,89 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 02:49:16,89 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,89 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,89 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,89 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,89 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,89 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,106 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 02:49:16,106 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,107 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 02:49:16,107 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,107 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,109 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:16,109 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 02:49:16,109 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:16,109 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:16,109 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 02:49:16,109 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:16,109 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:16,109 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:16,109 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 02:49:16,109 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 02:49:16,109 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:16,109 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:16,109 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,109 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,109 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:16,109 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:16,111 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:49:16,111 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:49:16,111 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-3-shard-people-config_testChangeMemberVotingStatusForShard, member-2-shard-people-config_testChangeMemberVotingStatusForShard] 02:49:16,111 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:16,112 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:16,112 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:49:16,112 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 02:49:16,112 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 02:49:16,112 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 02:49:16,112 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 02:49:16,112 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Candidate to Leader 02:49:16,113 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 02:49:16,113 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,113 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,113 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 02:49:16,113 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 02:49:16,113 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 02:49:16,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 02:49:16,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:16,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 02:49:16,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:16,162 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 02:49:16,166 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 02:49:16,166 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,166 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 02:49:16,166 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,166 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,168 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:16,168 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 02:49:16,168 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:16,168 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:16,168 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 02:49:16,168 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:16,168 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,168 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:16,168 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:16,168 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 02:49:16,168 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:16,168 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:16,168 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 02:49:16,168 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:16,168 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,168 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:16,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:49:16,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:49:16,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testChangeMemberVotingStatusForShard, member-3-shard-cars-config_testChangeMemberVotingStatusForShard] 02:49:16,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:16,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:16,169 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:49:16,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 02:49:16,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 02:49:16,170 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 02:49:16,170 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 02:49:16,170 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Candidate to Leader 02:49:16,170 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 02:49:16,170 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 02:49:16,170 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 02:49:16,171 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,171 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 02:49:16,171 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 02:49:16,171 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 02:49:16,172 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,172 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 02:49:16,172 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 02:49:16,172 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 02:49:16,172 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 02:49:16,172 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-310557986] 02:49:16,173 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 02:49:16,173 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:16,173 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:49:16,173 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 02:49:16,173 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-563325511] 02:49:16,173 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:16,173 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-563325511] 02:49:16,174 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#879042657] 02:49:16,174 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:49:16,174 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-1022513670] 02:49:16,174 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-1022513670] 02:49:16,174 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#941759962] 02:49:16,174 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:49:16,174 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-2060303948] 02:49:16,174 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-2060303948] 02:49:16,175 PM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard cars: {member-2=false, member-3=false} 02:49:16,175 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 02:49:16,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:49:16,177 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,177 PM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]], state: Idle 02:49:16,177 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,177 PM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard)] 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]], state: Idle 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING]] 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]] 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=5ef95a41-dce3-4ae4-8ab1-864fda52f1b0}, logIndex: 0, isSendImmediate: true 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Setting last applied to 0 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] has been successfully replicated to a majority of followers 02:49:16,179 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 02:49:16,180 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 02:49:16,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 02:49:16,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Number of entries to be appended = 1 02:49:16,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:49:16,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 02:49:16,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 02:49:16,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=VOTING]] 02:49:16,180 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 02:49:16,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Log size is now 1 02:49:16,180 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Number of entries to be appended = 1 02:49:16,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Commit index set to 0 02:49:16,180 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 02:49:16,180 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 02:49:16,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:16,180 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 02:49:16,180 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatusForShard} from local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:49:16,180 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 02:49:16,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Setting last applied to 0 02:49:16,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 02:49:16,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 02:49:16,180 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING]] 02:49:16,180 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Log size is now 1 02:49:16,180 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Commit index set to 0 02:49:16,180 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully changed member voting states for shard cars 02:49:16,180 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 02:49:16,180 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 02:49:16,180 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Setting last applied to 0 02:49:16,180 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 02:49:16,181 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 02:49:16,181 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 02:49:16,181 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 02:49:16,181 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 02:49:16,181 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 02:49:16,181 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 02:49:16,181 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 02:49:16,181 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: 0, nextIndex: 1 02:49:16,181 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:16,181 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: 0, nextIndex: 1 02:49:16,181 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:16,184 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-310557986] 02:49:16,184 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 02:49:16,184 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-563325511] 02:49:16,184 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-563325511] 02:49:16,184 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#879042657] 02:49:16,184 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 02:49:16,184 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-1022513670] 02:49:16,184 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-1022513670] 02:49:16,184 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#941759962] 02:49:16,184 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 02:49:16,184 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-2060303948] 02:49:16,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-2060303948] 02:49:16,185 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 02:49:16,185 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,185 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,185 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 02:49:16,185 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 02:49:16,185 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 02:49:16,186 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,186 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,186 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 02:49:16,186 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 02:49:16,186 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 02:49:16,186 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 02:49:16,186 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 02:49:16,186 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,186 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 02:49:16,186 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,186 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,187 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 02:49:16,188 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:16,188 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:16,188 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 02:49:16,188 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 02:49:16,188 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:16,188 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:16,188 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,188 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:16,188 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:16,188 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 02:49:16,188 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 02:49:16,188 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,188 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:16,188 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,188 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:16,188 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 02:49:16,188 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 02:49:16,188 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,188 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:16,188 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:16,188 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,188 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 02:49:16,189 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 02:49:16,189 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,189 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 02:49:16,190 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 02:49:16,190 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:49:16,190 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:49:16,190 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 02:49:16,190 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testChangeMemberVotingStatusForShard, member-3-shard-people-oper_testChangeMemberVotingStatusForShard] 02:49:16,190 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:16,190 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:16,190 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:49:16,190 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 02:49:16,190 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 02:49:16,190 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 02:49:16,190 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 02:49:16,191 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 02:49:16,191 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:49:16,191 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:49:16,192 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,192 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 02:49:16,192 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 02:49:16,193 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 02:49:16,193 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:16,196 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:16,196 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:16,196 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:16,196 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 02:49:16,196 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,196 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 02:49:16,196 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,196 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:16,196 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,198 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 02:49:16,202 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 02:49:16,202 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 02:49:16,202 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 02:49:16,202 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 02:49:16,202 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 02:49:16,202 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 02:49:16,202 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,202 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 02:49:16,202 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 02:49:16,202 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 02:49:16,203 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, member-2-shard-cars-oper_testChangeMemberVotingStatusForShard] 02:49:16,203 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:16,203 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:16,203 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 02:49:16,203 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 02:49:16,203 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 02:49:16,203 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 02:49:16,203 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 02:49:16,203 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 02:49:16,203 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 02:49:16,203 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-cars-oper_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,203 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 02:49:16,203 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 02:49:16,203 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard status sync done false 02:49:16,204 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 02:49:16,204 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:16,213 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 02:49:16,213 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,213 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,213 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 02:49:16,213 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 02:49:16,213 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 02:49:16,214 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:16,214 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Stopping Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 02:49:16,214 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 02:49:16,214 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 02:49:16,214 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 02:49:16,214 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 02:49:16,215 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 02:49:16,215 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,215 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,215 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 02:49:16,215 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 02:49:16,215 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 02:49:16,215 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:16,215 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:16,215 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 02:49:16,216 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 02:49:16,216 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 02:49:16,216 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 02:49:16,217 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:49:16,217 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:49:17,27 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 02:49:17,207 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-4361887230769247356] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:49:17,238 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1902556325510241739] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 02:49:17,240 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:17,240 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 02:49:17,241 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:49:17.243] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 02:49:17,248 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 02:49:17,249 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:17,249 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:17,249 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 02:49:17,249 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 02:49:17,249 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Initiating leader transfer 02:49:17,249 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 02:49:17,249 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Initiating leader transfer 02:49:17,250 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@68c4bde3 02:49:17,250 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@8570677 02:49:17,250 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@68c4bde3 02:49:17,250 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@8570677 02:49:17,250 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 02:49:17,250 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 02:49:17,250 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 02:49:17,250 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 02:49:17,250 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 02:49:17,250 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:17,250 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:17,250 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 02:49:17,250 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 02:49:17,250 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:17,706 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer aborted 02:49:17,706 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 456.6 ms 02:49:17,706 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In unpauseLeader 02:49:17,706 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 02:49:17,706 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 02:49:17,707 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 02:49:17,707 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 02:49:17,767 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer aborted 02:49:17,767 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 517.3 ms 02:49:17,767 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In unpauseLeader 02:49:17,767 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 02:49:17,767 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 02:49:17,767 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 02:49:17,767 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 02:49:17,768 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 02:49:17,768 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 02:49:17,768 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 02:49:17,768 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:17,769 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:17,769 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 02:49:17,769 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 02:49:17,769 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Initiating leader transfer 02:49:17,769 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 02:49:17,769 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Initiating leader transfer 02:49:17,769 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 02:49:17,769 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 02:49:17,769 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@65aef398 02:49:17,769 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@65aef398 02:49:17,769 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 02:49:17,769 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 02:49:17,769 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:17,769 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2ac51fc7 02:49:17,769 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2ac51fc7 02:49:17,769 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 02:49:17,769 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:17,769 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 02:49:17,769 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:17,769 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 02:49:18,107 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 02:49:18,107 PM [cluster-test-shard-dispatcher-34] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 337.8 ms 02:49:18,107 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 02:49:18,107 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 02:49:18,107 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 02:49:18,107 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 02:49:18,108 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 02:49:18,117 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 02:49:18,117 PM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 348.0 ms 02:49:18,117 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 02:49:18,117 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 02:49:18,117 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 02:49:18,118 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 02:49:18,118 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 02:49:18,118 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 02:49:18,118 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 02:49:18,119 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 02:49:18,119 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 02:49:18,122 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/08/2025 14:49:18.124] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config